Speeches Shim
About the DEC
The Development Experience Clearinghouse (DEC) is the largest online resource for USAID funded technical and project materials for international development, with assets dating back to 1945. You are able to access information on over 190,000 USAID technical and program documents with more than 155,500 available for electronic download and growing everyday.
The purpose of the DEC is to strengthen USAID's development projects, activities, and programs by making these development experience documents available to USAID offices and mission staff, PVO's, NGO's, universities and research institutions, developing countries, and the public worldwide.
About the API
The DEC API can assist you in extracting meta-data and records from the DEC. The DEC website (https://dec.usaid.gov), with its rich user interface is optimized for search and discovery, but not the extraction of large amounts of data. Through the website, you can explore the DEC's offerings and become familiar with the content and search syntax of the DEC. After a certain amount of exploration, you may discover that you need to extract large amounts of data. That's where the DEC API comes in. It is designed to provide a mechanism to retrieve large amounts of data.
This page will provide basic instructions on how to call and interact with the inputs and outputs of the DEC API. In order to understand the offerings of the API, we expect that you have some experience with programming including constructing an URL, web methods calls, and http requests.
The API consist of three web points that provide distinct mechanisms to interface with the DEC. The points of access are qsearch.ashx, queries.svc, and request.axpx. Each contains their own methods to retrieve information from the DEC.
About the Dataset
The dataset includes the meta-data associated with each document record in the database. The meta-data is derived from the content of the medium and provides information such as the object's URL, title, publication date, authors, document types, and geographical regions. Please refer to the data dictionary for more information on what is returned by the API.
DEC Disclaimer
USAID provides data on the USAID Open Data Listing website ("this website") to the public to further USAID's values of transparency, participation, and collaboration in tangible ways that benefit the American people. The data may be reproduced or otherwise used freely, unless identified as being subject to copyright protection. If data is identified as being subject to a copyright, it is the obligation of the user to obtain permission from the original source to use or copy the data from the original source. USAID asks that the "USAID Open Data Listing" be cited as the source of the data used from this website.
USAID is providing this information as a public service and makes every effort to provide accurate and complete information. USAID makes no guarantee that there will be no errors, and makes no claims, promises or guarantees about the accuracy, completeness, or adequacy of the data on this website, and expressly disclaims liability arising from use, reference to, or reliance on any data contained on this website. This is a service that is continually under development, and USAID will make every effort to keep this site current and to correct errors brought to our attention. Errors and data requests are best reported via USAID’s Github account. USAID's open data team can be contacted at opendata@usaid.gov.
The data posted on this website may contain hypertext links or pointers to information created or maintained by other public and private organizations. These links and pointers are provided to enhance the public's information experience on the website. USAID does not control or guarantee the accuracy, relevance, timeliness, or completeness of any linked information. USAID make no claims, or guarantees about the accuracy and completeness of external data connected to this website, and expressly disclaims liability for errors or omissions.
The views and opinions of authors expressed herein do not necessarily represent the official views and opinions of USAID.
qsearch.ashx
Qsearch will immediately return the result set in the specified format as a downloadable feed for search results under 2,000 records. Any queries more than the record limit will return a "result set too large error."
Response Format:
CSV, JSON, XML
Response Type:
HTTPS GET
Authentication:
None
Required Variables:
q= String - Base64 URL Encoded Search Query for the DEC Database. If the field is ignored then qsearch will redirect back to help.html.
To form a valid Base64 URL encoded string, use these sites to assist in building your query.
"Search string" -> Base64 -> URLEncode -> Parameter -> Request -> Output
"Search string" -> U2VhcmNoIHN0cmluZw== -> U2VhcmNoIHN0cmluZw%3D%3D -> q=U2VhcmNoIHN0cmluZw%3D%3D -> https://dec.usaid.gov/api/qsearch.ashx?q=U2VhcmNoIHN0cmluZw%3D%3D
Optional Variables:
rtype= String Enumeration - Can be either CSV, JSON, or XML. Default is XML. If an invalid entry is supplied then the application will use default output type of XML.
Enumeration -> Parameter -> Output
CSV -> rtype=CSV -> https://dec.usaid.gov/api/qsearch.ashx?q=U2VhcmNoIHN0cmluZw%3D%3D&rtype=CSV
How to build your query:
Lets get Evaluations from the DEC using the search notation (documents.web_collection:("recent evaluations")) and output as a CSV file.
Take (documents.web_collection:("recent evaluations")) and convert it to Base64.
Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ== and URL encode it.
Take KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D and set as a 'q' parameter for the URL: https://dec.usaid.gov/api/qsearch.ashx
https://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D will return its results using its default output format as XML.
Append the string with the parameter and value notation of &rtype=CSV.
https://dec.usaid.gov/api/qsearch.ashx?q=KGRvY3VtZW50cy53ZWJfY29sbGVjdGlvbjooInJlY2VudCBldmFsdWF0aW9ucyIpKQ%3D%3D&rtype=CSV will now output as a CSV file.
Now you have a URL that will get all of the DEC's meta-data for the Evaluations as a CSV file.
Queries.svc
Queries.svc is a Mircosoft Windows Communication Foundation Service that is hosted by IIS. It provides an object to call methods that can return recourd counts, results below 2,000 records, and request larger sets in the same way Request.aspx provides. These methods can be referenced in an .NET application. Search the web for more information on WCF Services.
Response Type:
Mircosoft Windows Communication Foundation Service
Authentication:
None
Methods:
int CountResults(string email, string query, string version)
Counts the number of records in the search query. If the query does not parse or there was an internal error then zero(0) is returned.
email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.
Returns the number of records found in the DEC using the query string.
string Request(string email, string query, string version, string format)
Appends the request into the API's processing queue and sends a confirmation email on the success of the request.
email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.
format - Can be either CSV, JSON, or XML. If the value does not parse then XML is assumed.
Returns the confirmation or error text of the transaction.
string Search(string email, string query, string version, string format)
Searches the DEC for an immediate response. If the record count is larger than 2,000 records then a "result set too large error." is returned.
email - requestor's email address
query - DEC search string
version - Reserved for future releases. Set to NULL the value is ignored.
format - Can be either CSV, JSON, or XML. If the value does not parse then XML is assumed.
Returns the string representation of the result set in the output format provided. See the Output Samples for more information.
Request.aspx
Request.aspx will request and record the requestor's email address, search query, and output format and when the submit button is clicked, it appends the request into the API's processing queue. Immediately the site will send an email to the requestor's address to provide a confirmation of the request. Then during the site's off peek hours, the API will generate the result and send an additional email to the requestor with a link to the data in the requested format as a downloadable file from the site.
Links are stored on the server for 30 days. They are not renewed or refreshed automatically and its content stays the same. If you need to refresh the data then you should complete a new request. If you need a link to our site to remain pernment then contact the DEC webmaster at decwebmaster@usaid.gov and submit your request in the email.
Response Type:
Forms response
Authentication:
Requires email address to process the request.
Form fields:
Email *
Required - Requestor's email address to send the confirmation and result link to when the operation is completed.
Search Query *
Required - Search Query for the DEC Database
Check query
Optional function to evaluate your search query to a count of the records. Use this to make sure that your search query is going to return an expected number of results.
Output Format
Defaults to XML - A dropdown box that allows the user to choose a result format between XML, JSON, and CSV.
Submit
Checks the query for valid values and if they are valid then enters the values to the queue for processing after hours. Two emails will be generated afterward submittion. The first will be send immediatly to the requestor and the second will be send when the process is complete.
Reset
Reset the form fields for a new query. Note that Reset does not clear the confirmation or error text in the form.
Output
The page will return a confirmation text and a confirmation email to the user. You can use the confirmation number to track insidents with the API to the decwebmaster at decwebmaster@usaid.gov
Result:
The result depends upon the user specified "rtype" or output type of XML, JSON, or CSV. The schema contains the search term, the number of records found, and a listing of all matching records in the database.
Sample CSV Return
"Sample Value","Another Sample Value | And a multi-value"\n
| Sample Property | Another Sample Property |
|---|---|
| Sample Value | Another Sample Value | And a multi-value |
Note: Multivalues in the CSV format is delimited by space then pipe and then space again. (' | ').
Sample JSON Return
"SearchTerm": "Sample",
"SearchDate": "2014-01-01T11:59:59.0000000-04:00",
"RecordsFound": "68",
"Records": [
{
"Sample Property":
{
"value": ["Sample Value"]
},
"Another Sample Property":
{
"value": ["Another Sample Value", "And a multi-value"]
},
...
},
{
},
...
]
}
Sample XML Return
<SearchTerm>Sample</SearchTerm>
<SearchDate>2014-01-01T11:59:59.0000000-04:00</SearchDate>
<RecordsFound>68</RecordsFound>
<Records>
<Record>
<Property name="Sample Property">
</Property>
<Property name="Another Sample Property">
<Value>And a multi-value</Value>
</Property>
...
</Record>
<Record>
</Record>
...
</Records>
</Search>
Data Dictionary
Header information
searchTerm
searchDate
RecordsFound
| Property name | Data type | Sample value | Description |
|---|---|---|---|
| searchTerm | String | Forward Evaluations | Text value of the search query executed against the DEC. |
| searchDate | Date | 2014-01-01T11:59:59.0000000-04:00 | Exact date and time the search query started its execution against the DEC. |
| RecordsFound | Integer | 68 | Indicates the number of records found with the search query and the number of record nodes in the output. |
Column ID's
Abstract
Ancillary_Data
Bibliographic_Type
ContentType
Contract_Grant_Number
Credit
Date_Resource_Created
Description
Descriptors_Topical
Descriptors_Geographic
File
File_Size
Inst_Author
Inst_Publisher
Inst_Sponsor
Language
Mime_Type
New_Thesaurus_Terms
Notes
Personal_Author
Primary_Subject
Publication_Date_Freeform
Related_Doc_Links
Report_Number
Series_Title
Title
Title_Translated
Unique_ID
URI
USAID_Geography
USAID_Project_Number
| Property name | Data type | Sample value | Description |
|---|---|---|---|
| Abstract | String | Summarizes mid-term evaluation of a project... | The summary of the artifact. |
| Ancillary_Data | String | Period of report: August 2003-July 2004 | Can be anything from the document that doesn't fit in a predefined field; also information that is not available in the document. |
| Bibliographic_Type | String | Reference Document | Bibliographic Code and Name; Indicates the type of artifact. |
| ContentType | String | Documents | Identifies the content type of the record. Can be Documents, Audio, Video, etc. |
| Contract_Grant_Number | String | PCE-I-00-00-00000-00 | Number of the A.LD. contract, grant, PASA, or RSSA which funded the production of the artifact. |
| Credit | String | Director: James Cameron | The people who are responsible for creating the object: i.e. photographer, director, writer, interviewer, interviewee, actor, etc. |
| Date_Resource_Created | Date | 1/1/1986 12:00:00 AM | Displayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; defined format. |
| Description | String | Video is in color and contains sound | Describes the artifact for its look in feel. Relates to Multimedia content types. |
| Descriptors_Topical | String | HIV/AIDS | Field contains primary descriptors which reflect the subject content of the artifact. |
| Descriptors_Geographic | String | Latin America | Geopolitical descriptors from the USAID thesaurus. This is the geographic topic of the asset. |
| File | URL | http://pdf.usaid.gov/pdf_docs/PNAAA000.pdf | The actual artifact document. |
| File_Size | String | 300 KB | The file size in KB. |
| Inst_Author | String | 000035 - U.S. Agency for International Development (USAID) | Authoring organization name. |
| Inst_Publisher | String | 012781 - USAID. Knowledge Services Center (KSC) | Publishing organization name. |
| Inst_Sponsor | String | 000035 - U.S. Agency for International Development (USAID) | Sponsoring organization name. |
| Language | String | English | Identifies the language of the multimedia file. Identifies the language of the text of the document being analyzed. If more than one language is used, enter the appropriate codes in alphabetical order, separated by a comma and space. |
| Mime_Type | String | application/pdf | Mime type; the internet media type identifier. |
| New_Thesaurus_Terms | String | Interim evaluation | Terms NOT included in the"A.I.D. Thesaurus" are entered separately here to describe concepts not adequately covered by existing descriptors. Field also contains proposed descriptors. |
| Notes | String | Additional information can be found at... | Area for freeform notes. |
| Personal_Author | String | Shah, Rajiv | Persons responsible for authorship, includes editors and compilers. |
| Primary_Subject | String | Development organizations | Primary Subject; A major subject category is assigned to all technical document records entered after August 1985 and to all project documents entered after February 1987. The subject is derived from the "A.I.D. Thesaurus" list of Major Subjects. |
| Publication_Date_Freeform | String | Spring 1987 | Displayed publication date on the document; Date the document was (is to be) published; submission of dissertation or thesis; unstructured format. |
| Related_Doc_Links | String | http://pdf.usaid.gov/pdf_docs/PNAAA001.pdf | List of other documents that relate to this artifact. |
| Report_Number | String | AS-0-0 | Alphanumeric number assigned to a document by the creators. |
| Series_Title | String | Population technical assistance project occasional paper, no. 6 | Records the title of the publication series and the part. |
| Title | String | USAID HIV/AIDS segment : overview -- final | The title and subtitle of an artifact. |
| Title_Translated | String | USAID HIV/AIDS segment : overview -- final | English translation of the Document Title. |
| Unique_ID | String | PN-AAA-000 | Unique identifier. Also used to indicate that the cataloguer has selected a particular document for cataloguing since they are the ones applying the DocID. PDs are for non-technical documents, PNs technical, and PCs are non-USAID documents, typically found on the web. |
| URI | URL | https://dec.usaid.gov/dec/content/Detail.aspx?ctID=ODVhZjk4NWQtM2YyMi00Y... | URL to the details record page in the DEC. |
| USAID_Geography | String | Ecuador | Geographic Area Name as defined by USAID; Related to code field. This is where the funding for the asset came from. |
| USAID_Project_Number | String | 1234567 | Concatenation of the Project and Sub-project numbers. Seven or Nine digits. |
Comment
Make a general inquiry or suggest an improvement.