Web Crawler Demo

A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). wiki

  1. Web Crawler
    1. 환경구성
    2. Chart Crawler
    3. 멜론차트 (Top50)

Web Crawler


환경구성

구분구성비고
수집차트멜론일간차트https://www.melon.com/chart/day/index.htm
웹크롤러heroku/node.jshttps://get-chart.herokuapp.com/
클라이언트html5/jQueryhttp://127.0.0.1:4000/blog/2020-09-12-melon-chart/
  • node.js CORS (cross origin resource sharing) 문제 해결방법

    방안해결방안코드
    1Access-Control-Allow-Origin response header 추가res.header("Access-Control-Allow-Origin", "*");
    2CORS middleware 추가app.use(cors());
  • Access-Control-Allow-Origin response header 추가
    app.get('/data', (req, res) => {
      res.header("Access-Control-Allow-Origin", "*");
      res.send(data);
    });
    
  • CORS middleware 추가
    const express = require('express');
    const cors    = require('cors');
    const app     = express();
    
    const corsOptions = {
      origin: 'http://localhost:3000', 
      credentials: true, 
    };
    app.use(cors(corsOptions)); 
    

Chart Crawler

https://get-chart.herokuapp.com/

멜론차트 (Top50)


© 2018. All rights reserved.

Powered by Hydejack v8.4.0