Written byAsmit JoshionFebruary 16, 2018
In our last post, we showed you how to automate your data delivery process in the Grepsr app.
This time let’s have a quick look at data feeds and endpoints[*]. Your scraped data’s Endpoint API is the final stop it makes in its journey— starting from the host website, then to your Grepsr account via our crawler, and finally on to your website or database. The routes available within Grepsr for your data to make its way to you from the app are:
- Endpoint API
- RSS Feed
An API is simply a URL that represents your data as an object or a collection of objects.
The Overview section gives you descriptions of certain basic concepts.
In each of the following sections, when you click Test Method, the app gives you a URL to call the Endpoint API with a preview of your data in JSON format, as shown in the stats section below.
In the /v1/report/dataset section, you can view your dataset from a specific crawl run, which you can select from the dropdown in the
history_id form field.
The /v1/report/dataset/time section gives you similar functionality as mentioned above, with the added capability of specifying time frames.
The /v1/report/stats section gives you the overall stats for the current report, as shown above.
If you haven’t activated your JSON feed yet, you’ll see an Activate JSON button.
Once you activate, the app gives you a URL to your JSON data with a preview below it.
Through RSS feeds, you can keep track of your data as they’re being scraped by the app.
Once you fill out the form and update your feed, content at your end will be updated in real-time as our crawler gathers each record.
For each of the Item Title, Item Description, Item Link and Category fields, select the relevant column names from the dropdown.
Furthermore, you can toggle between the form’s Normal and Advanced modes clicking the button at the form’s top right corner.
These Endpoint API, JSON and RSS Feeds functionalities give you quick access to your data without having to download any files. You can simply call the URLs mentioned above in your web browser to retrieve your data from our servers.
So go ahead and set it up — simplify and streamline your content gathering process.
*The test scrape in this case is off books.toscrape.com, which is a handy resource for web scraping practice.