256 Kilobytes

Scripting and Automation Articles

Type ArticlesContent written by the staff, as well as hand-picked user content Category Scripting and AutomationInformation and discussion of scripting of various types. Tag All

Profile Photo - August R. Garcia

Spit in the Face of God: Five Tasks You Didn’t Know Could be Done with Google Sheets

Articles in Spreadsheets | By August R. Garcia

Published 8 months agoTue, 12 Feb 2019 07:53:13 -0800 | Last update 7 months agoThu, 28 Feb 2019 22:37:26 -0800 📌

Profile Photo - Hash Brown

Last Reply Any opportunity to spit in the face of god is good by me. Excellent post homie.  Hash Brown,

Any opportunity to spit i...

Mon, 27 May 2019 07:59:54 -0700 4 months ago
🗨
2
🐏
2
👁
1,914
Profile Photo - Hash Brown

An introduction to scraping with Python and BeautifulSoup

Articles in Web Scraping, Data Analysis | By Hash Brown

Published 9 months agoTue, 08 Jan 2019 22:34:09 -0800 | Last update 7 months agoMon, 11 Mar 2019 22:19:33 -0700 📌

Profile Photo - August R. Garcia

Last Reply Very nice, as they say. Other scraping methods For those who don’t perhaps have the skills needed to code something there are also other... August R. Garcia,

Very nice, as they say.

Wed, 09 Jan 2019 03:16:31 -0800 9 months ago
🗨
1
🐏
1
👁
1,406
Profile Photo - August R. Garcia

How to Make a Simple, Automatically-Updating Dashboard for Python Scripts Using HTML, JavaScript, and JSONP

Articles in Scripting and Automation | By August R. Garcia

Published 1 month agoThu, 05 Sep 2019 04:39:30 -0700 | Last update 1 month agoThu, 05 Sep 2019 05:15:22 -0700

Profile Photo - Default

Last Reply Thank you for the post. Blogger at Radio Box https://www.youtube.com/watch?v=Jfh_qbzhUKI RadioBox,

Thank you for the post.

Sat, 21 Sep 2019 04:07:21 -0700 3 weeks ago
🗨
2
🐏
1
👁
262
Profile Photo - August R. Garcia

The Basics to Web Scraping with cURL and XPath

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 3 months agoFri, 28 Jun 2019 09:02:04 -0700 | Last update 3 months agoTue, 02 Jul 2019 17:37:33 -0700

Profile Photo - August R. Garcia

Last Reply Bump. Some more cURL, used to determine the quality of proxygo's proxies: for i in {1..48} ; do curl https://www.blackhatworld.com/seo/ssl-proxi... August R. Garcia,

Bump. Some more cURL, use...

Thu, 29 Aug 2019 18:31:09 -0700 1 month ago
🗨
4
🐏
2
👁
1,664
Profile Photo - August R. Garcia

[BASH, cURL] Yellow Pages Scraper: Fully Functional Script with Source Code

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 3 months agoFri, 05 Jul 2019 23:22:06 -0700 | Last update 3 months agoSat, 06 Jul 2019 01:44:02 -0700

Profile Photo - August R. Garcia

What a nice, free YellowPages scraper.

MoreEdit: When trying to scrape indefinitely (~100+ pages), there's some buggy behavior with exit conditions currently. If/when an updated script is poste...
🗨
0
🐏
1
👁
303
Profile Photo - August R. Garcia

Downloading Bulk Images: ThisPersonDoesNotExist with Python and urllib2

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 7 months agoThu, 14 Mar 2019 06:25:36 -0700 | Last update 7 months agoThu, 14 Mar 2019 08:05:08 -0700

Profile Photo - August R. Garcia

Last Reply Here's a shorter version with cURL and BASH that does basically the same thing: for i in $( seq 1 10 ) ; do curl --user-agent "Some User-Agent St... August R. Garcia,

Here's a shorter version...

Thu, 04 Jul 2019 12:56:36 -0700 3 months ago
🗨
2
🐏
0
👁
3,101
Profile Photo - August R. Garcia

[cURL, BASH] How to Crawl and Scrape DuckDuckGo Search Results

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 3 months agoTue, 02 Jul 2019 17:29:24 -0700 | Last update 3 months agoThu, 04 Jul 2019 19:21:52 -0700

Profile Photo - August R. Garcia

You can use these same concepts to build...

MoreAs discussed recently, it is relatively easy to scrap various arbitrary pieces of data using cURL (and XPath). You can use these same concepts to buil...
🗨
0
🐏
1
👁
591
Profile Photo - August R. Garcia
Profile Photo - August R. Garcia

What a nice trick.

MoreWhat a nice trick. How to Extract Emails from HTML with Google Sheets Code function get_raw_html(url) { // The code below logs the H...
🗨
0
🐏
0
👁
314
Profile Photo - August R. Garcia

[Infographic] The Beginner's SQLite Cheat Sheet

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 5 months agoSat, 04 May 2019 23:59:37 -0700

Profile Photo - August R. Garcia

The important hotkeys, commands, and tri...

MoreWhat a great infographic. Copy-Pasteable Version of the SQLite Commands Cheat Sheet General SQLite Commands and Information Opening SQLit...
🗨
0
🐏
0
👁
283
Profile Photo - August R. Garcia

Analyzing the Web: Downloading the Majestic Million, Setting up SQLite, Crawling the Web, and Generating Reports

Articles in Web Scraping, Data Analysis | By August R. Garcia

Published 5 months agoWed, 24 Apr 2019 03:29:27 -0700 | Last update 5 months agoThu, 25 Apr 2019 09:14:10 -0700

Profile Photo - August R. Garcia

Last Reply The longest domain names in the Majestic Million, most of which are expired and basically all of which are terrible garbage: 255461  ... August R. Garcia,

The longest domain names...

Mon, 29 Apr 2019 09:10:47 -0700 5 months ago
🗨
2
🐏
2
👁
458
Profile Photo - August R. Garcia

Generating Fake Reddit Comments with Markov Chains

Articles in Scripting and Automation | By August R. Garcia

Published 7 months agoThu, 28 Feb 2019 16:31:18 -0800 | Last update 7 months agoThu, 28 Feb 2019 17:23:39 -0800

Profile Photo - Some Guy

Last Reply Yeah that should be doable. The only thing would be that you would have to make sure those KWs are in the word_dict so that there are pairs to generat... Some Guy,

Yeah that should be doabl...

Tue, 05 Mar 2019 01:25:03 -0800 7 months ago
🗨
4
🐏
1
👁
1,145
Profile Photo - August R. Garcia

Custom Capitalization in Google Sheets -- Handle Acronyms, Industry Terms, Etc. in Titles

Articles in Spreadsheets | By August R. Garcia

Published 8 months agoFri, 01 Feb 2019 23:06:44 -0800 | Last update 8 months agoFri, 01 Feb 2019 23:12:43 -0800

Profile Photo - Some Guy

Last Reply Bonus note. If the value in F2 does not contain any question marks, add a question mark at the end; otherwise, get the unmodified string in F2:... Some Guy,

Bonus note. If the value...

Fri, 01 Feb 2019 23:46:51 -0800 8 months ago
🗨
1
🐏
0
👁
303
Profile Photo - August R. Garcia
Profile Photo - August R. Garcia

Sometimes, you have to extract emails an...

More form.article-form { border:1px solid black; border-radius:8px; box-shadow:2px 2px rgba(70,70,70,0.2); padding:0.5em 0.75em; background-color:rgba(170...
🗨
0
🐏
1
👁
851
Profile Photo - Hash Brown
Profile Photo - August R. Garcia

Last Reply This whole process is a classic in a ton of Internet marketing work. Definitely worth experimenting with the process for anyone new (or even moderatel... August R. Garcia,

This whole process is a c...

Wed, 02 Jan 2019 02:59:05 -0800 9 months ago
🗨
1
🐏
1
👁
499