PATTERN AND DATA ANALYSIS OF WEB CRAWLED DATA USING JSON PARSER AND ASSOCIATION MINING

ABSTRACT

With the advancements in the internet, searching web is much significant. To retrieve the web pages automatically, web crawler is used. Web crawler feeds on a seed URL and visits all the subsequent URLs to gather information. The processed information is stored in the documents with a file format of file.json known as JSON documents. Further, to find the relationships between web pages, association rule mining is used. The frequent items are found using Apriori algorithm. Association rules are formed using these frequent items. In this text a crawler is proposed that crawls an arbitrary website. Then from the structured data of JSON file, association rules are predicted. These association rules are of much significance and can be used to obtain data and solve queries in many desktop as well as web applications.

FULL TEXT