有些网站的文章列表是api请求的，这种网站也是可以爬取的
如这个网站 http://www.tiyuxiu.com/
<br/>
在检查器中看到api为
<br/>
开始写代码
```~~~
package main
import (
"encoding/json"
"github.com/PeterYangs/article-spider/apiSpider"
"github.com/PeterYangs/article-spider/fileTypes"
"github.com/PeterYangs/article-spider/form"
"github.com/PeterYangs/tools"
"log"
"strconv"
)
func main() {
f := form.Form{
Host: "http://www.tiyuxiu.com",
Channel: "/data/list_0_[PAGE].json?__t=16192263",
Limit: 5,
PageStart: 1,
DetailFields: map[string]form.Field{
"title": {Types: fileTypes.SingleField, Selector: "body > div.container.main-container.clear.clearfix > div.pleft.mt10 > div.article-header > h1"},
"content": {Types: fileTypes.HtmlWithImage, Selector: "#main-content"},
"desc": {Types: fileTypes.Attr, Selector: "meta[name=\"description\"]", AttrKey: "content"},
"keyword": {Types: fileTypes.Attr, Selector: "meta[name=\"keywords\"]", AttrKey: "content"},
},
DetailMaxCoroutine: 1,
ApiConversion: func(result string) []string {
var jsons []map[string]interface{}
err := json.Unmarshal([]byte(result), &jsons)
if err != nil {
log.Print(err)
return []string{}
}
var linkList []string
for _, m := range jsons {
linkList = append(linkList, m["url"].(string))
}
return linkList
},
HttpHeader: map[string]string{"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36"},
}
apiSpider.Start(f)
}
```
和常规爬取不同的是，需要写好**ApiConversion**转换函数，也就是要将获取的api列表数据解析，并返回一个 **[]string** 的文章详情链接数据
