How to Scrape Images on Any Site

Posted by taewoo on June 3, 2011 0 comments

1) Auto-click with imacro on FireFox. Make sure u give it enough time to “sleep” between iterations.
2) Save the cached images onto your disk (use CacheViewer)

Kachiiiing!

Poor Man’s Google Scrape Technique

Posted by taewoo on March 2, 2009 0 comments

How to scrape for links on Google using notepad..

This requires cygwin or some sorta shell interpreter with grep/sed capability.

1) Search for whatever you wish.  Make sure you get 100 results per page (under “advanced search”)

2) Copy paste the entire screen

  • Ctrl+A  - this selects all text
  • Ctrl+C – copies text onto buffer
  • Open Notepad (or your fav. text editor), press Ctrl + V – pastes text onto editor
  • Save (i.e. urls.txt).

3) Run this command on the file

 grep “Cached” urls.txt | sed -e ‘s/ – .*//g’ -e ‘s/\?.*//g’ -e ‘s/^      //g’ | sort | unq

 

Voila

Firefox caching iframe data

Posted by taewoo on July 10, 2008 9 comments

Ok this is a repost cuz of stinkin’ DreamHost.

If you have an auto-generated iframe from a javascript, and this iframe has dynamic data, FireFox (but not IE) wll cache your data. Even if you have

<META HTTP-EQUIV="Cache-Control" CONTENT="no-cache">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">

After days of pulling out my hair (or what little I have left), i found out the only way around this is by calling this in your JavaScript:

iframe.src = iframe.src;

This basically forces the browsers to reload. Not the most efficient solution but it works. Also, I haven’t tested to see if I could remove this with FireFox 3 coming out but no matter. The rest of the world is gonna take 1-2 years to download FF3 anyway.

PS: Hairloss is not due to masturbation. Thank god for that.