Web Farming from the CI Perspective 

Doug House, president of the Washington Information Group, gave an insightful talk on finding information on private companies, ones that are hidden from obvious information resources. Within the US and worldwide, these companies have less reporting requirements to governmental agencies and hence less publicly available information recorded about them. There are over 250,000 companies worldwide, of which 94% are these hidden companies.

As House elaborated upon his methodology, he offered several assertions about CI research in general.

First, secondary research (i.e., scanning printed directories and querying online databases) contributes only 30% of the needed information in a typical investigation. Further, secondary information consists of descriptions of the past rather than intent toward the future. The most relevant information comes from primary research, especially finding and talking to the ‘right’ individuals via telephone interviews.

Second, most information from secondary research is not online (or even web-based). It is in printed material. Even when information is available online, it is usually quicker and easier to find and digest as printed material.

Third, 80% of the needed information is available internally within your company. This information is not necessarily in operational systems or data warehouses, but in the heads and file cabinets of people throughout the organization. From sales reps to executives, these persons receive daily nuggets of valuable information about customers, competitors, suppliers, and so forth. Unfortunately, this information is rarely managed in any systematic way.

Fourth, systematic intelligence requires a mature understanding of strategic objectives, which rarely is the case. Doug remarked that you know you have a strategy when you walk away from revenue because that revenue is not part of your strategy. For the most part, companies are constantly searching for their strategies. Therefore, they are unable to implement effective intelligence systems. Systems that are implemented without clear objectives are a big waste of money and time.

These four assertions have sufficient validity to force us (web farmers) to seriously ponder them. In particular, in what ways should we modify the methodology and architecture of web farming, assuming these assertions are valid? Here are some suggestions.

First, the information analyst (as described in Chapter 4, 5, and 6 of the WF book) should cultivate CI skills and employ them in the information refining process. In particular, the use of printed material and telephone interviews should be a standard part within the investigative or detective work of information refining. The relationship should be synergism. Web-based resources should lead one to specific printed material and specific individuals. Likewise, printed material and phone interviews should lead one to specific web sites for background material.

Second, the infrastructure supporting the information analyst should be enhanced. If we take the first suggestion seriously, then the analyst should have a workbench that integrates with the telephone (e.g., logging, auto-dialing, audio recording, note-taking). Further, the workbench should accommodate hard-copy materials (e.g., scanner, OCR, digital camera), along with a nearby bookshelf of critical (printed) directories.

Third, our definition of web-based information resources should be open-ended. We should avoid being myopic about web-based information. In particular, the web farming database should deal with interview notes, printed annual reports, and URL links with equal ease. In other words, we need to incorporate the capabilities of traditional document management.

Fourth, the “Getting Started” stage should be lengthened. In the WF book, I suggested that one could complete the requirements in a month or two. That may be the case. However, the requirements should include a clear statement of strategic objectives and a mature ability to execute those objectives. Few companies are there, implying that investments in web farming are at risk. In other words, the best indicator for successful web farming is the degree to which web farming is a natural deduction of a mature corporate strategy and a natural extension of existing strategic systems.

Fifth, the role of a variety of internal information resources should be reconsidered. It is obvious that internal email, Notes databases, and electronic documents of any kind are target resources. However, more attention should be paid to informal resources, such as observations and opinions of various persons within the organization. Online discussion forums on critical issues, periodic polling of specific groups on narrow topics, and just good ‘people-networking’ should be an integral capability of the web farming system.

Finally, there is an opposite twist to the CI perspective. As discussed, web farmers (as a new profession) can learn from our (older and wiser) CI colleagues. However, the opposite is also true. Amid the explosive changes in our global information environment, the CI profession is challenged to reinvent itself. Like research librarians and other professions, CI professionals often feel threatened by the Internet.

There are no doubts that in a few short years the bulk of mankind’s information will be available via the Web. There are no doubts that usage of this information has been awkward but will rapidly improve. There are no doubts that the companies who learn how to harness this potential will have a competitive advantage over those that do not.

The CI perspective can enrich our fledgling discipline of web farming. Likewise, web farming has the potential to enrich CI if an open and honest sharing occurs between the disciplines.

- Richard Hackathorn