Perl soared to popularity as a language for creating and managing web content, but with LWP (Library for WWW in Perl), Perl is equally adept at consuming information on the Web. LWP is a suite of modules for fetching and processing web pages. The Web is a vast data source that contains everything from stock prices to movie credits, and with LWP all that data is just a few lines of code away. Anything you do on the Web, whether it's buying or selling, reading or writing, uploading or downloading, news to e-commerce, can be controlled with Perl and LWP. You can automate Web-based purchase orders as easily as you can set up a program to download MP3 files from a web site. Perl & LWP covers:

• Understanding LWP and its design

• Fetching and analyzing URLs

• Extracting information from HTML using regular expressions and tokens

• Working with the structure of HTML documents using trees

• Setting and inspecting HTTP headers and response codes

• Managing cookies

• Accessing information that requires authentication

• Extracting links

• Cooperating with proxy caches

• Writing web spiders (also known as robots) in a safe fashion

Perl & LWP includes many step-by-step examples that show how to apply the various techniques. Programs to extract information from the web sites of BBC News, Altavista, ABEBooks.com, and the Weather Underground, to name just a few, are explained in detail, so that you understand how and why they work. Perl programmers who want to automate and mine the web can pick up this book and be immediately productive. Written by a contributor to LWP, and with a foreword by one of LWP's creators, Perl & LWP is the authoritative guide to this powerful and popular toolkit.

Amazon.com® Reader Reviews (Ranked by Helpfulness)

Average Amazon.com® Rating:  Based on 10 Ratings

This book can teach you expert-level web scraping/munging. - 2003-07-12
Reviewer Rating:
If you aren't yet comfortable using object-oriented Perl modules, the multitude of examples will at least allow you see how it's done even if you're a bit fuzzy on what's happening 'underneath' when you call object methods. If you're comfortable learning how to do something without knowing exactly why it works, then the author's clear step-by-step explantions and numerous progressively more powerful examples should make this book accessible even to relatively innexperienced Perl programmers.

More experienced programmers will understand better why things work, but any Perl programmer will set this book down feeling empowered to turn the web into their own valet. No longer do you need to check multiple sites looking for interesting information. Instead, you can readily author code to do that for you and alert you when items of interest are found. You can use these tools to free up personal time, to harvest information to inform business decisions, to automate tedious web application testing, and a zillion other things.

The author's clear exploration of the relevant Perl modules leaves the reader with a good depth of understanding of what these modules do, when you might want to use which module, and how to use them for real world tasks. Before reading the book, I knew of these modules, but they were a rather intimidating pile. I'd used a few of them on occasion for rather limited projects, but was reluctant to invest the time required to read all of the documentation from the whole collection. Mountains of method-level documentation do not a tutorial make. This book takes all of that information, selects the most important parts, and ensures that those parts are covered in progressively more powerful and/or flexible examples.

If you know Perl and you're sick of 'working the web' to get information and you want the web to work for you instead, then you need this book. I had a personal project that was on the back burner for a couple of years because it just sounded too hard. The weekend after I finished this book, I wrote what I had previously thought to be the hard part of that project and it was both easy and fun. This book makes hard things not just possible, but actually easy.

-matt

Terrible, bug-infested book... - 2002-11-05
Reviewer Rating:
I really don't know how the previous 5 reviews gave this book 5 stars. I was really excited about this book when I first read the reviews, and now here I am only a few chapters in and already thinking about dumping it altogether. This book has so many flaws for its size, the biggest of which was the codes. I am no Perl expert, but could find my way around in a decent size program. However, no examples I have tried so far in the book actually worked, and some of these are just 10-20 lines long. I am completely new to LWP, I guess like anyone who would buy this book, so it's hard for me to see what the author is doing. The explanation of the code didn't help much either. As oppose to explaining the steps, he just said "the code below does this". And it's pretty obvious little or no editing has gone into this book. If you do buy this book, you'll probably want to make a trip to the Errata page at the Oreilly website. The amount of typos, printing errors, warnings and grammatical mistakes found by readers and editors listed on this page rivals the usuable content of the book itself. You know what, I have spent way too much on this book already.....

Great book! - 2003-03-16
Reviewer Rating:
If you are unfamiliar with LWP and web scraping, or HTML parsing using tokens and trees, I strongly recommend this book. It's the best *introduction* to these topics I've been able to find. Sean's style is clear and concise-just what I expect from an O'Reilly book.

To get the most out of this book, you'll want to be familiar with Object Oriented programming in Perl, because (with the exception of LWP::Simple) all the modules discussed in this book use objects.

Also, don't expect the LWP sample code in the book to work correctly. Many of the sites that the scripts try to "scrape" have changed their layout since this book was published, braking the scripts. This isn't a problem though, because the samples Sean provides are very short and clear, so it's not necessary to run them in order to figure out how they work.

A Wonderful Book - 2006-08-18
Reviewer Rating:
I bought this book to get information automatically on japanese stocks(for example, charts, price, volume, PER, PBR, ROE, ROA, News, messages on Yahoo! Japan BBS for stocks) from the WEB every day.

Somehow this book has not yet translated into Japanese language.
I think this book would sell very well if translated into Japanese. Many demands.

This book is self-contained about the WEB, so you need little Perl programming rules and don't have to have knowledge on the Internet Protocols(HTTP) at all.

In most cases, all you need to do is to modify an example program on this book for your use very little.

Honest Assessment of Burke's Perl & LWP - 2007-07-13
Reviewer Rating:
This is not your typical clunker with endless pages of filler material. It gets right to the point. If you want to learn about using Perl to interact with the internet, this would be a good book to help you get there. I have purchased several Perl books that supposedly teach you how to write code for use with the internet, but they are difficult to understand, and most of the examples just don't work. This book is an exception to that trend. It is the only one I have found so far that has useable, workable examples. The subject matter is still challenging, but Burke is able to explain it enough to give you a clue. If you are looking for help in handling HTTP programmatically, then here is your book.

Browse Similar Topics

Top Level Categories:
Internet/Online
Programming

Sub-Categories:
Internet/Online > Scripting
Programming > Perl