0x80
/
rsstube
1
1
Fork 0
Code Issues 6 Pull Requests Projects Releases Wiki Activity

Cloudflare #3

New Issue
Open
opened 2021-11-09 13:14:10 -05:00 by 0x80 · 2 comments
0x80 commented 2021-11-09 13:14:10 -05:00
Owner
Copy Link

Cloudflare serves 403 errors for all requests from rsstube, meaning currently rsstube cannot support any site that uses Cloudflare.

The solution will probably require the following:

  • changing the user-agent string
  • additional HTTP headers
  • possibly providing some cookie
Cloudflare serves 403 errors for all requests from rsstube, meaning currently rsstube cannot support any site that uses Cloudflare. The solution will probably require the following: - changing the user-agent string - additional HTTP headers - possibly providing some cookie
0x80 commented 2021-12-30 01:48:34 -05:00
Author
Owner
Copy Link

57e2a92532 adds an --unbreak option (based on Tor Browser's request headers) for dealing with Cloudflare.

This seems to be resolved for now, but I'll leave this issue open indefinitely to track updates, as I expect the "unbreak" techniques will need to be periodically updated.

https://negativezero.link/code/0x80/rsstube/commit/57e2a92532c2b589f97d643326cb461f6a4ae0ff adds an `--unbreak` option (based on Tor Browser's request headers) for dealing with Cloudflare. This seems to be resolved for now, but I'll leave this issue open indefinitely to track updates, as I expect the "unbreak" techniques will need to be periodically updated.
0x80 commented 2022-04-02 19:01:09 -04:00
Author
Owner
Copy Link

Tor Browser requests Accept-Encoding: gzip, deflate, br. Right now, rsstube doesn't do this because I don't know how to then decompress the response data. I suspect this is part of why some Cloudflare sites aren't working over Tor, even when --unbreak is used and the site works in the Tor Browser.

TODO: If server sends compressed data, decompress that data.
TODO: Add Accept-Encoding: gzip, deflate, br HTTP header to --unbreak option.

Tor Browser requests `Accept-Encoding: gzip, deflate, br`. Right now, rsstube doesn't do this because I don't know how to then decompress the response data. I suspect this is part of why some Cloudflare sites aren't working over Tor, even when `--unbreak` is used and the site works in the Tor Browser. TODO: If server sends compressed data, decompress that data. TODO: Add `Accept-Encoding: gzip, deflate, br` HTTP header to `--unbreak` option.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: 0x80/rsstube#3
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?