NIUCLOUD是一款SaaS管理后台框架多应用插件+云编译。上千名开发者、服务商正在积极拥抱开发者生态。欢迎开发者们免费入驻。一起助力发展! 广告
如下例子 ``` package main import ( "github.com/PeterYangs/article-spider/fileTypes" "github.com/PeterYangs/article-spider/form" "github.com/PeterYangs/article-spider/spider" ) func main() { f := form.Form{ Host: "https://www.duote.com", Channel: "/sort/50_0_wdow_0_[PAGE]_.html", Limit: 5, PageStart: 1, ListSelector: "body > div.wrap > div.box > div.main-left-box > div > div.bd > div > div.soft-info-lists > div", ListHrefSelector: " a", DetailFields: map[string]form.Field{ "title": {Types: fileTypes.SingleField, Selector: "body > div.wrap.mt_5 > div > div.main-left-box > div.down-box > div.soft-name > div > h1"}, }, DetailMaxCoroutine: 1, } spider.Start(f) } ``` 结果为 ![](https://img.kancloud.cn/f8/d9/f8d94ffa700c4d4df7714cba36d08e86_483x616.png) <br/> 有些内容为空,可以用格式转换处理一下,修改一下代码 ``` package main import ( "github.com/PeterYangs/article-spider/fileTypes" "github.com/PeterYangs/article-spider/form" "github.com/PeterYangs/article-spider/spider" ) func main() { f := form.Form{ Host: "https://www.duote.com", Channel: "/sort/50_0_wdow_0_[PAGE]_.html", Limit: 5, PageStart: 1, ListSelector: "body > div.wrap > div.box > div.main-left-box > div > div.bd > div > div.soft-info-lists > div", ListHrefSelector: " a", DetailFields: map[string]form.Field{ "title": {Types: fileTypes.SingleField, Selector: "body > div.wrap.mt_5 > div > div.main-left-box > div.down-box > div.soft-name > div > h1", ConversionFormatFunc: conversion}, }, DetailMaxCoroutine: 1, } spider.Start(f) } func conversion(data string, resList map[string]string) string { if data == "" { return "微信" } return data } ``` **ConversionFormatFunc**,有两个参数,第一个参数是当前数据,第二个参数是所有结果的map,和web框架的获取器的原理类似