浏览代码

[fix] startpage engine : characters with diacritic were preceded by whitespace, and cleaner way to parse the result.

Dalf 11 年前
父节点
当前提交
a2928e8d83
共有 1 个文件被更改,包括 5 次插入6 次删除
  1. 5
    6
      searx/engines/startpage.py

+ 5
- 6
searx/engines/startpage.py 查看文件

19
     global base_url
19
     global base_url
20
     results = []
20
     results = []
21
     dom = html.fromstring(resp.content)
21
     dom = html.fromstring(resp.content)
22
-    for result in dom.xpath('//div[@class="result"]'):
22
+    # ads xpath //div[@id="results"]/div[@id="sponsored"]//div[@class="result"]
23
+    # not ads : div[@class="result"] are the direct childs of div[@id="results"]
24
+    for result in dom.xpath('//div[@id="results"]/div[@class="result"]'):
23
         link = result.xpath('.//h3/a')[0]
25
         link = result.xpath('.//h3/a')[0]
24
         url = link.attrib.get('href')
26
         url = link.attrib.get('href')
25
         parsed_url = urlparse(url)
27
         parsed_url = urlparse(url)
26
-        # TODO better google link detection
27
-        if parsed_url.netloc.find('www.google.com') >= 0:
28
-            continue
29
-        title = ' '.join(link.xpath('.//text()'))
30
-        content = escape(' '.join(result.xpath('.//p[@class="desc"]//text()')))
28
+        title = link.text_content()
29
+        content = result.xpath('./p[@class="desc"]')[0].text_content()
31
         results.append({'url': url, 'title': title, 'content': content})
30
         results.append({'url': url, 'title': title, 'content': content})
32
     return results
31
     return results