T77 - Scrape web data and transfer it to your database
|
Task Description |
Scrape web data and transfer it to your database. For e.g. scrape information from an online book database and build your database. This sample task uses an example from http://www.getabstract.com/summaries/27000000/onlineDate/all/paged/en/it-production-and-logistics.html just for illustrative purposes.
 T77.atmn  Books.mdb
|
| Solution |
 |
Record a new task
- Click on Tools->Web Recorder.
- Enter the starting URL for e.g. http://www.getabstract.com/summaries/27000000/onlineDate/all/paged/en/
it-production-and-logistics.html and click on Start button.
- Click on ‘Show all’ link.
- Extract list of books on webpage. For that, click on 'Extract Data' button in the Web Recorder toolbar.
- From Extract Data Option window, select 'Pattern Based Data' option and click on 'Next' button to start 'Extract Multiple Data' wizard.
- Step1 of wizard is to select the fields you need to extract. Click on 'Add' button and capture a part from list. You will be presented with a small window having 'Capture' button. Click on that 'Capture' button to capture another instance of same field in order to match the pattern. Specify the Column Name in 'Extract Data Field' window and save. Screenshot1.
- After you captured the fields, click on 'Next' link. In Step2, you can specify if data is in multiple pages. In this case, the data is in single page so you can skip this step and click on Next button.
- In Step3, specify the name and path of CSV file, where you need to save the extracted data for e.g. C:\Books.csv. You can view the data using 'Preview Data' button. Eventually, click on Finish button to complete the wizard. Screenshot2
- Stop the recording and save the task.
|
Screenshot 1
Screenshot 2
Screenshot 3
Screenshot 4
Screenshot 5 |
 |
Import the extracted data into your database.
- Open the recorded task in Task Editor from Tools-> Task Editor.
- After you extracted the book details at Line #3, connect to your database using Database:Connect command. Screenshot3
- Then insert ‘Read from ‘CSV/Text File’ command to open C:\Books.csv.
- After ‘Read from ‘CSV/Text File’ command, insert ‘Loop for each row in CSV/Text File’ command.
- Inside this loop block, insert Execute SQL query command to insert the data into database. Screenshot4
- Using ‘String Manipulation’ command you can trim the extracted data or extract the part of data extracted from web.
- Entire task will look as in Screenshot 5.
|
 |
How to run the task
- In case you do not have Automation Anywhere installed, please download and install it from, http://www.automationanywhere.com/download/freeTrial.htm
- Save T77.atmn under location, My Documents\Automation Anywhere\Automation Anywhere\My Tasks
- Copy Books.mdb under C:\
- Launch Automation Anywhere and from Task List select T77.atmn and click on RUN button to run the task.
|
|
| |
| When you run this task, it will extract the book details from a website into CSV file then will loop through all the extracted book details in CSV file and transfer it to the database. |
|
 |
| Wow! This program is fantastic. I was given the task to figure out how to get a list of over 1000 contacts off of a website and into Excel. I was trying to write some kind of macro for days through Excel's VB extension and was making no progress. So I went online and found Automation Anywhere, it took me only minutes to write a macro to copy the addresses. I owe Tethys Solutions hours of my life that I was prepared to spend copying and pasting. Thank you so much Tethys" |
- Ben Kinney, Monterey media
|
| Read Customer Testimonies >> |
|
