Sychronizing data extracted with Grepsr
Written by Subrat on May 23, 2012
One of Grepsr’s most powerful feature is the ability to synchronize data in a variety of different ways when a new data is available. The whole idea behind this feature is to automate our data delivery process. It would really be tedious if you have to login to your grepsr account everyday to check if there is new data. If you are comfortable doing that, no problem 🙂 but as a user we’d rather have it sync automatically.
I’ve already touched base on FTP and Dropbox in my earlier post. I’ll try to explain all our sync options and how you could use them in a real world scenario.
There are currently 5 data sync options. They are:
- Call Back URL
- Google Docs
Scenario #1: You run a shopping website and you need prices compared with your competitors once every day – Callback URL
You would want to use, Call Back URL option for this. Grepsr would post the latest data to your system via HTTP POST. You need to configure your callback URL, typically something like (http://www.yourdomain.com/callback.php) to read data posted to it. Your callback script would then import this data in your database (mySQL, MongoDB, MS SQL etc) and perform whatever operations you need to perform on the data. Perhaps match the product SKU with your own database and reduce your product’s price by 1% to stay competitive? Its your call!
Call Back URLs are ideal when there are a lot of data.
Scenario #2: You are a news reporter and you need latest news from various sources once you wakeup every morning – DropBox or Email or Google Docs
So you are not a technical guy, but use DropBox to stay in sync with your files? Thats all that you need. You can have us gather news data from various sources once every day or less. Then just attach your DropBox account with our service and that is all! Grepsr will send you the news as Spreadsheets. You can then process the news and re-write them or blog them! We even deliver the news as PDFs if that helps.
If you need to use Email, thats fine too. Grepsr will send you the same files in the emails which you can download to view. The files are securely hosted in our cloud, so you need not worry about data security.
If you are a big fan (who isnt?) and heavy user of Google Docs, you can have Grepsr automatically sync the latest data to your Google Docs account, so that the data is in one place. This method is ideal only when the volume of data is less than 5000 records, because Google Docs has its restrictions.
Scenario #3: You are a business owner, and you need to extract heavy amount of data and archive them for processing – FTP
FTP would be the right deal for this case. Grepsr can send all the data that was extracted directly to your FTP server, neatly organized in folders. You can then choose to process them either manually or using your in-house technologies. This method is ideal because it helps you keep archived data for future reference.
Some of our monthly customers have the habit of doing things manual. We just want to ask you to make full use of all these wonderful sync options we have in place! If you need help, you could always buzz us – we are ever ready to offer tips and advice on any of our features!