ddcurl on github.com, a web scraping wrapper
My first repository of Git (version control) on GitHub.com. Everything look good, compared with svn, git is so much easier. But I miss a good TortoiseSVN tool. I wish in future, git has similar tool like TortoiseSVN.
Now I use both. Git for open source project and svn for paid project.
Here is my codes that usually I use to scrape the web. It offer normal fetching (GET), form POST-ing, and login (maintain cookie and session). It doesn’t offer parsing method like DOM selection, regular expression or text splitting. That’s your problem .. hehe. Ah no .. it because each of websites has its own pattern, so it is impossible to generalize them.
Enjoy.
