大众点评反爬虫机制，用户评论自编码替换

本人是研究推荐系统的，最近在看到一篇文章Exploiting Ranking Consistency Principle in Representation Learning for Location Promotion觉得挺有意思的，想深入了解一下，然后想着从大众点评上爬点数据，于是就随手打开大众点评，点开评论，打开源码，研究一下爬虫策略，这才发现，现在大众点评除了在代理IP上下功夫反爬虫外，在源码上也做文章了，具体如下：

网页源码：

这<span class="fr-DMrt">span>店我<span class="fr-Ky0H">span><span class="fr-QmtS">span><span class="fr-SHzo">span><span class="fr-mUfe">span>烈<span class="fr-mAqU">span>call，<span class="fr-SHzo">span>我<span class="fr-OL5j">span>前在合肥吃过<span class="fr-QmtS">span>最好吃<span class="fr-QmtS">span>一<span class="fr-DMrt">span><span class="fr-AaxC">span>锅店了。<br />首先<span class="fr-Eg8Q">span><span class="fr-DMrt">span><span class="fr-QmtS">span>毛肚<span class="fr-Ky0H">span><span class="fr-QmtS">span>超棒，<span class="fr-BK2E">span><span class="fr-APZx">span><span class="fr-SHzo">span>能看<span class="fr-QmtS">span>出<span class="fr-gnD7">span><span class="fr-QmtS">span>棒，脆脆<span class="fr-QmtS">span>口感，有些<span class="fr-AaxC">span>锅店<span class="fr-QmtS">span>毛肚端上<span class="fr-gnD7">span><span class="fr-wbqv">span>显能看出......p> <a href="javascript:void(0);" class="more fold J-more">展开评论<i class="icon">i>a> div> <div class="info J-info-all clearfix Hide"> <p class="desc J-desc">这家店我真<span class="fr-QmtS">span><span class="fr-SHzo">span>强烈打call，<span class="fr-SHzo">span>我目前在合肥吃过<span class="fr-QmtS">span>最好吃<span class="fr-QmtS">span><span class="fr-dK0T">span>家火锅店了。<br />首先他家<span class="fr-QmtS">span>毛肚真<span class="fr-QmtS">span>超棒，<span class="fr-BK2E">span>量<span class="fr-SHzo">span>能看<span class="fr-QmtS">span>出来<span class="fr-QmtS">span>棒，脆脆<span class="fr-QmtS">span><span class="fr-2QAf">span>感，有些火锅店<span class="fr-QmtS">span>毛肚端上来明显能看出来发白<span class="fr-QmtS">span>痕迹，明显<span class="fr-SHzo">span>冻过<span class="fr-QmtS">span>，而它家<span class="fr-SHzo">span>冰鲜<span class="fr-ZTIr">span>理<span class="fr-QmtS">span>。<br />黄喉也<span class="fr-OnYW">span>好吃，虽然这个<span class="fr-BK2E">span>量<span class="fr-QmtS">span>黄喉合肥有好几家能做到，但<span class="fr-SHzo">span><span class="fr-wlIq">span>格没它家便宜<span class="fr-jT0z">span>！！胜在性<span class="fr-wlIq">span>比高<span class="fr-jT0z">span>！！<br />我<span class="fr-sMte">span>还点了<span class="fr-dK0T">span>份<span class="fr-xiFS">span>，哇塞，这个分量你<span class="fr-sMte">span>可以看图<span class="fr-dK0T">span>，分量真<span class="fr-QmtS">span><span class="fr-OnYW">span><span class="fr-72sB">span>，<span class="fr-xiFS">span>片被片得<span class="fr-OnYW">span>薄，在火锅里涮几十秒<span class="fr-O0dH">span>能吃了！<br /><span class="fr-dK0T">span>直觉得吃火锅<span class="fr-O0dH">span><span class="fr-SHzo">span>吃锅底，菜大同小异都差不多，我<span class="fr-sMte">span>点<span class="fr-QmtS">span><span class="fr-SHzo">span>魔鬼辣，但<span class="fr-SHzo">span>相信我，<span class="fr-dK0T">span>点都不辣！！！！！！中间<span class="fr-SHzo">span>有点药膳味道<span class="fr-QmtS">span>鸡汤？可能<span class="fr-SHzo">span>吧，白汤<span class="fr-O0dH">span><span class="fr-SHzo">span><span class="fr-OnYW">span>普通<span class="fr-QmtS">span>红枣香菇汤。。。涮菜<span class="fr-QmtS">span>话基本没用白汤。这样感觉有点浪费。强制鸳鸯！！！但辣汤真<span class="fr-QmtS">span><span class="fr-OnYW">span>好吃，不咸，不像有<span class="fr-QmtS">span>店越吃越咸！！！<br />服务<span class="fr-OnYW">span>好，<span class="fr-vsrl">span>有<span class="fr-ALv0">span>毛巾擦手之<span class="fr-10Lg">span>！服务<span class="fr-ALv0">span><span class="fr-2a4U">span>！<br />唯<span class="fr-dK0T">span>不太好<span class="fr-QmtS">span>，<span class="fr-O0dH">span><span class="fr-SHzo">span>卫生问题，可能我去<span class="fr-QmtS">span>时间太晚了，调料区挖葱<span class="fr-QmtS">span>时候小虫子乱飞，挺影响心<span class="fr-2a4U">span><span class="fr-QmtS">span>！<br />不管怎么说，<span class="fr-SHzo">span><span class="fr-vsrl">span>二刷<span class="fr-QmtS">span>店！p> <a href="javascript:void(0);" class="more unfold J-more">收起评论

截取其中一句：

这<span class="fr-DMrt">span>店我<span class="fr-Ky0H">span><span class="fr-QmtS">span><span class="fr-SHzo">span><span class="fr-mUfe">span>烈<span class="fr-mAqU">span>call，

可以发现其中有些字被网页元素替换了，加载到前端的时候再显示原始评论，可以说很灵性了。

 

转载于:https://www.cnblogs.com/bryanbao/p/9338304.html