Scraping FAQ

Help yourself answer your concerns with our frequently updated FAQ of Web scraping. Absolute originality is just a dream. Everyone of us scrape one way or another, though perhaps not so obvious.

What is Web scraping anyway?

Web scraping is the process you need to search and collect unstructured data on the web to put in more usable structures for other purposes such as content enrichment and market research.

As the definition implies, it generally involes 2 stages:

1. Search and collect unstructured web data or information bits
2. Store the bits and pieces into well designed semantic structures such as XML and relational tables for later retrieval that is versatile and cost-effective

How is Web scraping done for you?

A Web scraping project is initiated by declaring the project goal. It is based on this goal that we decide what data is most critical that has to be retrieved, what is secondary that can add desirable values but is not indispensable, and what is totally unnecessary that can or should be neglected.

Then we proceed to make a detailed scraping plan and draft data structure proposal for your inspection and possible revisions.

Upon your approval, the actual scraping is started. We will contact you immediately in case we see it necessary to adjust previous plans.

Why scraping has nothing to do with content duplication.

We scrape data in the most atomic way possible, that is in basic semantic triples that are straight facts. With millions of simple statements such as these, we are able to produce virtually unlimited possible combinations and arrangements of content that will come out absolutely unique.

Search engines will never see the results as duplicate content. Just think of it as we are writing an original article with all the words from our native language. Can the words be copyrighted or held as duplicate content even though everyone of us uses 'the'?

What is data semantics?

Semantics of data facilitates computer understanding of them, enabling total automated processing.

Well designed semantics for scraped data makes reuse much more easy, thus significantly reducing derivative costs but enriching content in many possible ways.

It's hard to reuse legacy, aged Web data in old HTML pages because they are poorly structured and absent of necessary semantics. This is exactly where we come in.

What is Web harvesting?

Similar to Web scraping, Web harvesting is the process to find, retrieve and then organize information.

Web harvesting tools and softwares get the job done by strictly following search patterns, neglecting many details that may prove to be critical to your original project goal. Therefore, in most cases, custom scraping service is more desirable than a mere tool, not to mention the entire learning curve and time spent to use it effectively.

What is Unstructured Data Management?

Unstructured Data Management refers to the retrieval and organization of data that is not previously in a structured format. It also involves the management techniques necessary to make use of large collection of unstructured data.