256 Kilobytes

All Content > cURL

Type All Category All Tag cURL

Profile Photo - August R. Garcia

[BASH, cURL] Yellow Pages Scraper: Fully Functional Script with Source Code

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 2 weeks agoFri, 05 Jul 2019 23:22:06 -0700 | Last update 2 weeks agoSat, 06 Jul 2019 01:44:02 -0700

Profile Photo - August R. Garcia

What a nice, free YellowPages scraper.

MoreEdit: When trying to scrape indefinitely (~100+ pages), there's some buggy behavior with exit conditions currently. If/when an updated script is poste...
🗨
0
🐏
1
👁
81
Profile Photo - August R. Garcia

[cURL, BASH] How to Crawl and Scrape DuckDuckGo Search Results

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 3 weeks agoTue, 02 Jul 2019 17:29:24 -0700 | Last update 2 weeks agoThu, 04 Jul 2019 19:21:52 -0700

Profile Photo - August R. Garcia

You can use these same concepts to build...

MoreAs discussed recently, it is relatively easy to scrap various arbitrary pieces of data using cURL (and XPath). You can use these same concepts to buil...
🗨
0
🐏
0
👁
146
Profile Photo - August R. Garcia

The Basics to Web Scraping with cURL and XPath

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 3 weeks agoFri, 28 Jun 2019 09:02:04 -0700 | Last update 3 weeks agoTue, 02 Jul 2019 17:37:33 -0700

Profile Photo - August R. Garcia

Last Reply Basic DuckDuckGo result scraping: curl https://duckduckgo.com/html?q=asdf | awk '{$1=$1};1' | sed '/^$/d' | hxselect -c a.result__url | sed '/^$... August R. Garcia,

Basic DuckDuckGo result s...

Mon, 01 Jul 2019 15:56:40 -0700 3 weeks ago
🗨
3
🐏
2
👁
418
Profile Photo - August R. Garcia
Profile Photo - Some Guy

Last Reply Also, this tool is fairly useful: https://incarnate.github.io/curl-to-php/ Some Guy,

Also, this tool is fairly...

Sun, 14 Apr 2019 03:35:46 -0700 3 months ago
🗨
1
🐏
0
👁
114
Profile Photo - August R. Garcia
Profile Photo - August R. Garcia

Billing info is an array of one item.

MoreError: Cannot deserialize instance of `java.util.ArrayList` out of START_OBJECT token at line:1, column:301" Solution: Billing info is an arra...
🗨
0
🐏
0
👁
128