Automation Anywhere Call: 1-888-484-3535   

 
Automation AnywhereWeb Data Extraction
  Home > Product > Solutions > Web Data Extraction > T79  
T79 - Scrape unstructured data from the web and transfer it to Excel
Task Description

Scrape unstructured data from the web and transfer it to Excel. This sample task uses an example from http://www.partsgeek.com/catalog/2003/bmw/z4/air_intake/air_mass_meter.html just for illustrative purposes.

Automation Anywhere Task file T79.atmn

Note: Save the .atmn file in My Documents -> Automation Anywhere -> Automation Anywhere -> My Tasks

Solution
Record a new task

  • Click on Tools->Web Recorder.
  • Enter the starting URL for e.g. http://www.partsgeek.com/catalog/2003/bmw/z4/air_intake/air_mass_meter.html and click on Start button.
  • Extract list of Related Parts from left panel on webpage. For that, click on 'Extract Data' button in the Web Recorder toolbar.
  • From Extract Data Option window, select 'Pattern Based Data' option and click on 'Next' button to start 'Extract Multiple Data' wizard.
  • Step1 of wizard is to select the fields you need to extract. Click on 'Add' button and capture a part from list for e.g. ‘Air Filter’. You will be presented with a small window having 'Capture' button. Click on that 'Capture' button to capture another instance of same field in order to match the pattern for e.g. ‘Air Mass Meter’. Screenshot0
  • Specify the Column Name in 'Extract Data Field' window and save. Screenshot1
  • After you captured the field, click on 'Next' link. In Step2, you can specify if data is in multiple pages. In this case, the data is in single page so you can skip this step and click on Next button.
  • In Step3, specify the name and path of CSV file, where you need to save the extracted data for e.g. C:\Parts.csv. You can view the data using 'Preview Data' button. Eventually, click on Finish button to complete the wizard.
  • Click on any Part under ‘Related Parts’ say ‘Air Pump’.

Screenshot 1



Screenshot 2



Screenshot 3



Screenshot 4



Screenshot 5



Screenshot 6
Extract unstructured data

  • To extract unstructured Part details on right side of webpage, click on 'Extract Data' button in the Web Recorder toolbar.
  • From Extract Data Option window, select 'Pattern Based Data' option and click on 'Next' button to start 'Extract Multiple Data' wizard.
  • In Step1 of wizard capture/add number of fields you want to extract data from webpage (Screenshot 2). For e.g. Part Name, Part Number and Price. Screenshot 3
  • After you captured all the fields, click on 'Next' link. In Step2 click on Next button.
  • In Step3, specify the name and path of CSV file, where you need to save the extracted data say C:\PartDetails.csv. Click on Finish button to complete the wizard.
  • Stop the recording and save the task.
Edit the task to extract unstructured data for all the items (Parts) in list

  • Open the recorded task in Task Editor from Tools-> Task Editor.
  • After you extracted the list of Parts at Line #2, insert ‘Read from ‘CSV/Text File’ command to open C:\Parts.csv.
  • After ‘Read from ‘CSV/Text File’ command, insert ‘Loop for each row in CSV/Text File’ command.
  • Inside this loop block, put commands that clicks on Part link and extracts the Unstructured data.
  • Double-click on command that clicks on Part link and replace the actual Part with system variable, $Filedata Column(1)$. Save the command. Screenshot 4
  • Entire task will look as in Screenshot 5.

How to run the task:

  • In case you do not have Automation Anywhere installed, please download and install it from, http://www.automationanywhere.com/download/freeTrial.htm
  • Save T79.atmn under location, My Documents\Automation Anywhere\Automation Anywhere\My Tasks
  • Copy Data.xls under C:\
  • Launch Automation Anywhere and from Task List select T79.atmn and click on RUN button to run the task.
 
When you run this task, it will extract the required data (unstructured) from the web page into CSV file.
Buy Now Free Trial

Next Steps
Compare versions Compare versions
Watch a flash demo Watch a flash demo
Contact me Contact me
Ask the Expert Ask the Expert
Request live demo Request live demo

Call 1-888-484-3535

25,000+ Customers
25,000+ Customers
Customer list | Case Studies

Wow! This program is fantastic. I was given the task to figure out how to get a list of over 1000 contacts off of a website and into Excel. I was trying to write some kind of macro for days through Excel's VB extension and was making no progress. So I went online and found Automation Anywhere, it took me only minutes to write a macro to copy the addresses. I owe Tethys Solutions hours of my life that I was prepared to spend copying and pasting. Thank you so much Tethys"
- Ben Kinney, Monterey media

Read Customer Testimonies >>
  Products     Download     Purchase     Services     Support     Customers     Company     Sitemap     Trademark FacebookTwitterAutomation Anywhere Blog © Automation Anywhere 2010 Privacy Policy
View Demo View Demo Web Data Extraction Pattern Matching