Scrapy和Splash爬取iframe标签里的内容

Lua脚本

function main(splash, args)
  splash:go(args.url)
  splash:wait(args.wait)
  splash:runjs("iframe = function(){ 
    var f = document.getElementById('g_iframe'); 
    return f.contentDocument.getElementsByTagName('body')[0].innerHTML; 
  }")
  local result = splash:evaljs("iframe()")
  return result
end

配置好scrapy和splash后，就可以爬取到iframe标签里的内容了

github地址 https://github.com/ggsgit/NetEaseCloudMusic