Skip to content

added SplashHtmlResponse (Fixed #114) #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed

added SplashHtmlResponse (Fixed #114) #115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
68acbe9
added SplashHtmlResponse
atultherajput Mar 27, 2017
4b97884
fixed test_middleware
atultherajput Mar 27, 2017
2d87834
fixed test_middleware1
atultherajput Mar 27, 2017
044e6e9
fixed response
atultherajput Mar 27, 2017
62e50d6
fixed response1
atultherajput Mar 27, 2017
b688e62
fixed response2
atultherajput Mar 27, 2017
d5554e6
added as subclass of HtmlResponse
atultherajput Mar 28, 2017
b043b1f
now support application/xhtml+xml and application/vnd.wap.xhtml+xml
atultherajput Mar 28, 2017
6725240
removed unused import
atultherajput Mar 28, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion scrapy_splash/response.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import base64
import re

from scrapy.http import Response, TextResponse
from scrapy.http import Response, TextResponse, HtmlResponse
from scrapy import Selector

from scrapy_splash.utils import headers_to_scrapy
Expand Down Expand Up @@ -70,6 +70,14 @@ def replace(self, *args, **kwargs):
return _SplashResponseMixin.replace(self, *args, **kwargs)


class SplashHtmlResponse(SplashTextResponse,HtmlResponse):
"""
This HtmlResponse subclass sets response.url to the URL of a remote website
instead of an URL of Splash server. "Real" response URL is still available
as ``response.real_url``.
"""
pass

class SplashJsonResponse(SplashResponse):
"""
Splash Response with JSON data. It provides a convenient way to access
Expand Down
6 changes: 3 additions & 3 deletions scrapy_splash/responsetypes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@

class SplashResponseTypes(ResponseTypes):
CLASSES = {
'text/html': 'scrapy_splash.response.SplashTextResponse',
'text/html': 'scrapy_splash.response.SplashHtmlResponse',
'application/atom+xml': 'scrapy_splash.response.SplashTextResponse',
'application/rdf+xml': 'scrapy_splash.response.SplashTextResponse',
'application/rss+xml': 'scrapy_splash.response.SplashTextResponse',
'application/xhtml+xml': 'scrapy_splash.response.SplashTextResponse',
'application/vnd.wap.xhtml+xml': 'scrapy_splash.response.SplashTextResponse',
'application/xhtml+xml': 'scrapy_splash.response.SplashHtmlResponse',
'application/vnd.wap.xhtml+xml': 'scrapy_splash.response.SplashHtmlResponse',
'application/xml': 'scrapy_splash.response.SplashTextResponse',
'application/json': 'scrapy_splash.response.SplashJsonResponse',
'application/x-json': 'scrapy_splash.response.SplashJsonResponse',
Expand Down