Trapping Topic-Based Information - Information Trapping: Real-Time Research on the Web - page 102

In the last chapter, we looked at trapping internal information—informa-
tion such as who's linking to your site, or who's reading your RSS feed.
While that kind of trapping has its place, especially if you're trying to
make a living from your Web site, a broader use for information trapping
lies in trapping external, topic-based information. You might want to use
the external information you trap to create content for your site, pursue a
degree, educate yourself, and so on. he possibilities for the kinds of topic
searching you can do are almost endless.
Unfortunately, so are the resources on the Internet! So now it's time to look
at some large categories of information that you can monitor, from really
broad, like general search engines, to more narrow, like government sites.
Along the way, I'll provide some hints for searching particular kinds of sites
and some things to keep in mind when you're building queries. In later
chapters, we look at other trapping options, including multimedia sources.
For the most part, your largest data pools will come from general search
engines, and your smallest pools will derive from very focused sites like city
and state resources. Because of the diferent kinds of information ofered at
diferent types of sites, you have to build your traps a bit diferently for each
resource. But the underlying theory is the same: complete but manageable.
Monitoring General Search Engines
General search engines, the original foundation for information search on
the Internet, are starting to show their age. hey're large, unstructured,
chaotic pools of information that can be really tough to trap. On the other
hand, they are the original foundation for Internet search and therefore
people who have no idea what an RSS feed is, or how to get listed in a spe-
cialty search engine, will make really sure that they're listed in a general
search engine. And that, in a nutshell, is why you should consider trapping
in them, even though getting the perfect query together and monitoring
results can be a bit frustrating.
Google
General search engines are by deinition going to have very large data pools.
And Google's no exception. Google (google.com) has stopped providing a
page count for its search engine, but the last time it did, Google had over