`, and uses that to form `[domain]/feed/channel/[channel_id]`.
+
+Sample URLs:
+
+- https://invidio.us/watch?v=UEQfaTntFNw -> https://invidio.us/feeds/videos.xml?channel_id=UCo3AxjxePfj6DHn03aiIhww
+- http://axqzx4s6s54s32yentfqojs3x5i7faxza6xo3ehd4bzzsg2ii4fv2iid.onion/watch?v=4jYsu5-TJQ8 -> http://axqzx4s6s54s32yentfqojs3x5i7faxza6xo3ehd4bzzsg2ii4fv2iid.onion/feeds/videos.xml?channel_id=UCFn5CIjfAj-KZgJtsXw349g
+- https://invidio.us/channel/UCqC_GY2ZiENFz2pwL0cSfAw -> https://invidio.us/feed/channel/UCqC_GY2ZiENFz2pwL0cSfAw
+
+### Mastodon
+Mastodon user pages are at `[domain]/@[username]`, and posts are at `[domain]/@[username]/[postID]`. Mastodon user pages can also be found at `[domain]/users/[username]` (I think for compatability with Pleroma). The RSS feed for a given user is at `[domain]/@[username].rss`.
+
+Sample URLs:
+
+- https://mastodon.social/@gargron -> https://mastodon.social/@gargron.rss
+
+### Nitter
+Nitter is like Invidious, but for Twitter. It has [several public instances](https://github.com/zedeus/nitter/wiki/Instances). Nitter feeds have URLs at `[nitter domain]/[username]/rss`.
+
+Note, Twitter is not supported.
+
+Sample URLs:
+
+- https://nitter.net/doctorow -> https://nitter.net/doctorow/rss
+- http://3nzoldnxplag42gqjs23xvghtzf6t6yzssrtytnntc6ppc7xxuoneoad.onion/doctorow/status/1214288190317592576#m -> http://3nzoldnxplag42gqjs23xvghtzf6t6yzssrtytnntc6ppc7xxuoneoad.onion/doctorow/rss
+
+### Pleroma
+Pleroma user URLs are at `[domain]/users/[username]`. The Atom feed for a given user is at `[domain]/users/[username]/feed.atom`. This may not work for all instances.
+
+Sample URLs:
+
+- https://kawen.space/users/314 -> https://kawen.space/users/314/feed.atom
+
+### Plume
+Some Plume pages directly link to the Atom feed in an alternate link. `rsstube` first looks for this link: ``. If it doesn't find the link, it tries to create it. The feed for a given user is at `[domain]/@/[username]/atom.xml`. If the provided URL is a user's page (`[domain]/@/[username]`), it just puts the `atom.xml` on the end. If it's a blog post (`[domain]/~/[username]/[post-title]/`), it looks for a link to the user page: `` and appends the `atom.xml` to that.
+
+Sample URLs:
+
+- https://blog.funkwhale.audio/@/funkwhale -> https://blog.funkwhale.audio/@/funkwhale/atom.xml
+- https://fediverse.blog/~/MusingsByRg/firefox-add-ons-for-the-active-user -> https://fediverse.blog/@/rgx/atom.xml
+
+### Podcasts
+A podcast is just an RSS feed, but a lot of people post links to specific services like Apple Podcasts, instead of just posting the feed link. `rsstube` supports the following podcast sites, so you should be able to put in the URL for a podcast, and it will return the RSS feed URL.
+
+Basically, `rsstube` just looks on the page for a link to the actual RSS file.
+
+Sites with sample URLs:
+
+- Apple Podcasts (https://podcasts.apple.com/us/podcast/irl-online-life-is-real-life/id1247652431 -> https://feeds.mozilla-podcasts.org/irl)
+- Castro.fm (https://castro.fm/podcast/b3891201-4d76-4152-adeb-ae3e13d17f29 -> https://www.smashingsecurity.com/rss)
+- fyyd (https://fyyd.de/podcast/shades-of-brown/0 -> https://twoshadesofbrown.com/feed/podcast/)
+- Google Podcasts (https://podcasts.google.com/?feed=aHR0cHM6Ly9wb2RjYXN0Lm5lYXJseW9ucmVkLmNvbS9mZWVkLnhtbA -> https://podcast.nearlyonred.com/feed.xml)
+- Player.fm (https://player.fm/series/tea-with-queen-and-j -> http://feeds.soundcloud.com/users/soundcloud:users:83464467/sounds.rss)
+- Pocket Casts (https://pca.st/iHi8 -> https://feeds.simplecast.com/ZL7iUDiH)
+
+### Reddit
+Reddit URLs are extremely hackable, so `rsstube` assumes the user wants a feed for the actual URL provided, not specifically a user or a subreddit. To get an RSS feed for a Reddit page, just add `/.rss` to the end of the URL.
+
+Sample URLs:
+
+- https://old.reddit.com/r/privacy/ -> https://old.reddit.com/r/privacy/.rss
+- https://www.reddit.com/r/onions+tor+onionlovers/ -> https://www.reddit.com/r/onions+tor+onionlovers/.rss
+
+### Soundcloud
+Soundcloud users have built-in RSS feeds, which `rsstube` returns; **however, by default tracks are not added to them.** The feed comes in the form: `https://feeds.soundcloud.com/users/soundcloud:users:[user_id]/sounds.rss`
+
+To get the user ID, we download the page and search for the string `soundcloud:users:[user_id]"`
+
+Unfortunately, most Soundcloud RSS feeds will be empty, unless they are for a serialized podcast or something like that.
+
+Sample URLs:
+
+- https://soundcloud.com/greatnessgd -> https://feeds.soundcloud.com/users/soundcloud:users:2097827/sounds.rss
+- https://soundcloud.com/user-98066669 -> https://feeds.soundcloud.com/users/soundcloud:users:261098918/sounds.rss
+
+### Tumblr
+Tumblr provides an RSS feed for a given page at `[page URL]/rss`. For example, the feed for `username`'s posts will be: `https://username.tumblr.com/rss`. If you wanted all of `username`'s posts that were tagged `funny`, you could use `https://username.tumblr.com/tagged/funny/rss`
+
+(Note: Tumblr does not support HTTPS on all pages, so the URL may actually start with `http://` )
+
+Sample URLs:
+
+- https://william-snekspeare.tumblr.com/ -> https://william-snekspeare.tumblr.com/rss
+- https://william-snekspeare.tumblr.com/tagged/comic -> https://william-snekspeare.tumblr.com/tagged/comic/rss
+
+### Vimeo
+The Vimeo RSS feed for a user comes in the form: `https://vimeo.com/user[user_id]/videos/rss`
+
+If `rsstube` receives a URL in the form `https://vimeo.com/user[user_id]/` it can just append `videos/rss` to the end. If it receives a video URL, it will need to download the video page and find the user ID. It can be found in various parts of the page, including by searching for `creator_id`
+
+Sample URLs:
+
+- https://vimeo.com/user51602410 -> https://vimeo.com/user51602410/videos/rss
+- https://vimeo.com/63502573 -> https://vimeo.com/user2617788/videos/rss
+
+### Write.as
+See [WriteFreely](#writefreely).
+
+### WriteFreely
+WriteFreely(/Write.as) puts an RSS feed at `[user page]/feed/`. For single-user instances, this means `[domain]/feed/`, and for multi-user instances, this means `[domain]/[username]/feed/`.
+
+The user's page also lists the feed directly in a link element: ``
+
+Sample URLs:
+
+- https://tomskee.dev/ -> https://tomskee.dev/feed/
+- https://tomskee.dev/claiming-2020-for-myself -> https://tomskee.dev/feed/
+- https://wordsmith.social/rgx/ -> https://wordsmith.social/rgx/feed/
+- https://wordsmith.social/rgx/is-it-important-which-instance-you-are-in -> https://wordsmith.social/rgx/feed/
+
+### YouTube
+The YouTube RSS feed comes in the form: `https://www.youtube.com/feeds/videos.xml?channel_id=[channel_id]`
+
+Some YouTube channels have a URL in the form: `https://www.youtube.com/channel/[channel_id]`. In this case, we just take the channel ID from the end of the URL. In the case of `https://www.youtube.com/user/[username]` we can instead use `https://www.youtube.com/feeds/videos.xml?user=[username]`. For a playlist (`https://www.youtube.com/playlist?list=[playlist_id]`) we can use `https://www.youtube.com/feeds/videos.xml?playlist_id=[playlist_id]`.
+
+If the channel's URL is not in this form (e.g., if the user passes in a video URL), we download the page and try to get the channel ID from the metadata. YouTube seems to have moved this metadata info to JSON that's parsed by the JavaScript. `rsstube` now looks for the string: (`"channelId":"[channel_id]"`). If it's a user/channel page that for some reason wasn't already covered, we can also just use the canonical link to the /channel/ page (``).
+
+Sample URLs:
+
+- https://www.youtube.com/channel/UCqC_GY2ZiENFz2pwL0cSfAw -> https://www.youtube.com/feeds/videos.xml?channel_id=UCqC_GY2ZiENFz2pwL0cSfAw
+- https://www.youtube.com/watch?v=XzIXc5CTC2M -> https://www.youtube.com/feeds/videos.xml?channel_id=UCexJsljKV3x4x8p0eyTrjVg
+- https://www.youtube.com/watch?v=fFlfxwZDFzY&list=PL8D8D4240EC972114 (playlist - put this in quotes) -> https://www.youtube.com/feeds/videos.xml?playlist_id=PL8D8D4240EC972114
+
+## Unsupported Sites
+
+These sites are explicitly unsupported because they do not seem to support RSS natively. For these sites, you might have better luck with a tool like [RSS Box](https://github.com/stefansundin/rssbox) that can be run from your own server (or may have public instances) to create the RSS feed.
+
+Links here are not guaranteed to be accurate but are included for some reference of why I believe the given site does not support RSS natively.
+
+- Bandcamp [x](https://old.reddit.com/r/BandCamp/comments/6c47bh/question_no_more_rss_feeds/) [x](https://old.reddit.com/r/BandCamp/comments/9h2kcu/rss_reader_for_bandcamp_feed/)
+- Meetup [requires API requests to be authenticated](https://www.meetup.com/meetup_api/auth/).
+- Instagram [x](https://stackoverflow.com/questions/14747870/instagram-public-rss-feed)
+- TikTok
+- Twitch [x](https://discuss.dev.twitch.tv/t/twitch-rss-feed-link/1105)
+- Twitter [x](https://twittercommunity.com/t/does-api-1-1-no-longer-support-rss-feeds/11257)
diff --git a/docs/version b/docs/version
new file mode 100644
index 0000000..01692f5
--- /dev/null
+++ b/docs/version
@@ -0,0 +1 @@
+2020.01.10
diff --git a/install b/install
new file mode 100755
index 0000000..4faa6f1
--- /dev/null
+++ b/install
@@ -0,0 +1,70 @@
+#!/bin/sh
+
+ERR='\033[0;31m'
+WARN='\033[0;33m'
+NC='\033[0m'
+
+path=`dirname "$(readlink -f "$0")"`
+
+if [ $EUID -ne 0 ]; then
+ echo -e "${ERR}In order to install this utility, root permissions must be granted.${NC}"
+ echo "This script will add files to the following directories:"
+ echo "/usr/local/bin/"
+ echo "/usr/local/share/man/man1/"
+ echo "/etc/"
+ echo ""
+ echo 'Please run as root by switching to superuser first or running as "sudo install"'
+ echo "Alternatively, you can manually install this utility by symlinking $path/rsstube in /usr/local/bin/ and $path/rsstube.1 in /usr/local/share/man/man1/"
+ echo "Afterwards, run mandb as root to update your manual database."
+ echo "Copy the templates/config file to /etc/rsstube/"
+ exit
+fi
+
+execpath=/usr/local/bin/rsstube
+manpath=/usr/local/share/man/man1/rsstube.1
+file_problem=""
+
+if [[ -e $execpath ]]
+then
+# rm $execpath
+ echo -e "${WARN}Error: $execpath already exists.${NC}"
+ echo "If it was added from a version of rsstube < 2.0.0, then please delete it before installing:"
+ echo "sudo rm $execpath"
+ file_problem="true"
+fi
+
+if [[ -e $manpath ]]
+then
+# rm $manpath
+ echo -e "${WARN}Error: $manpath already exists.${NC}"
+ echo "If it was added from a version of rsstube < 2.0.0, then please delete it before installing:"
+ echo "sudo rm $manpath"
+ file_problem="true"
+fi
+
+if [[ -e /usr/bin/rsstube ]]
+then
+ echo -e "${WARN}Error: /usr/bin/rsstube exists and has a conflicting name.${NC}"
+ echo "Please delete /usr/bin/rsstube:"
+ echo "sudo rm /usr/bin/rsstube"
+ file_problem="true"
+fi
+
+if [[ -n $file_problem ]]
+then
+ echo -e "${WARN}Failed to install. Please fix above problems before trying again.${NC}"
+ exit
+fi
+
+
+ln -s $path/rsstube /usr/local/bin/rsstube
+ln -s $path/rsstube.1 /usr/local/share/man/man1/rsstube.1
+mandb
+if [[ ! -d /etc/rsstube ]]
+then
+ mkdir /etc/rsstube
+fi
+cp templates/config /etc/rsstube/config
+
+echo "rsstube is now installed. You can use it by typing:"
+echo "rsstube "
diff --git a/rsstube b/rsstube
new file mode 100755
index 0000000..6eaf7c8
--- /dev/null
+++ b/rsstube
@@ -0,0 +1,187 @@
+#!/bin/sh
+
+# rsstube - get RSS feed for YouTube channels and other sites
+# Copyright (C) 2019
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see Pleroma '
+)
+
+plume=(
+ ''
+)
+
+writefreely=(
+ ''
+ ''
+)
+
+for possible_site in ${supported_sites[@]}
+do
+ declare -n possible_patterns="$possible_site"
+ for pattern in "${possible_patterns[@]}"
+ do
+ site_pattern_match=`echo "$content" | grep -Pzo "$pattern" | tr '\0' '\n'`
+ if [[ -n "$site_pattern_match" ]]
+ then
+ site="$possible_site"
+ break
+ fi
+ done
+ if [[ -n "$site" ]]
+ then
+ break
+ fi
+done
diff --git a/scripts/http_errors b/scripts/http_errors
new file mode 100644
index 0000000..5a8d239
--- /dev/null
+++ b/scripts/http_errors
@@ -0,0 +1,89 @@
+cloudflare=""
+recaptcha=""
+
+# Cloudflare test
+cloudflare=`echo "$content" | grep "Attention Required! | Cloudflare "`
+if [[ -n "${cloudflare}" ]]
+then
+ error "Cloudflare blocked the request."
+
+ if [[ -n "${proxy}" ]]
+ then
+ warn "Are you using Tor? If so, try using rsstube again with the -t option."
+ fi
+fi
+
+# general Google reCAPTCHA
+recaptcha=`echo "$content" | grep "google.com/recaptcha/api"`
+if [[ -n "${recaptcha}" ]]
+then
+ error "reCAPTCHA requested"
+fi
+
+# HTTP errors
+http_response=`echo "$content" | head -1 | xargs`
+http_response=${http_response#* }
+
+# print appropriate status code
+case $http_response in
+ # 2xx success
+ 200) response_message="200 OK" ;;
+
+ # 3xx redirection should already be handled by curl -L flag
+ 301) response_message="301 Moved Permanently" ;;
+ 302) response_message="302 Found" ;;
+ 303) response_message="303 See Other" ;;
+ 307) response_message="307 Temporary Redirect" ;;
+ 308) response_message="308 Permanent Redirect" ;;
+
+ # 4xx client errors
+ 400) response_message="400 Bad Request" ;;
+ 401) response_message="401 Unauthorized" ;;
+ 402) response_message="402 Payment Required" ;;
+ 403) response_message="403 Forbidden" ;;
+ 404) response_message="404 Not Found" ;;
+ 405) response_message="405 Method Not Allowed" ;;
+ 406) response_message="406 Not Acceptable" ;;
+ 407) response_message="407 Proxy Authentication Required" ;;
+ 408) response_message="408 Request Timeout" ;;
+ 409) response_message="409 Conflict" ;;
+ 410) response_message="410 Gone" ;;
+ 411) response_message="411 Length Required" ;;
+ 412) response_message="412 Precondition Failed" ;;
+ 413) response_message="413 Payload Too Large" ;;
+ 414) response_message="414 URI Too Long" ;;
+ 415) response_message="415 Unsupported Media Type" ;;
+ 416) response_message="416 Range Not Satisfiable" ;;
+ 417) response_message="417 Expectation Failed" ;;
+ 418) response_message="418 I'm a teapot" ;;
+ 421) response_message="421 Misdirected Request" ;;
+ 422) response_message="422 Unprocessable Entity" ;;
+ 423) response_message="423 Locked" ;;
+ 424) response_message="424 Failed Dependency" ;;
+ 425) response_message="425 Too Early" ;;
+ 426) response_message="426 Upgrade Required" ;;
+ 428) response_message="428 Precondition Required" ;;
+ 429) response_message="429 Too Many Requests" ;;
+ 431) response_message="431 Request Header Fields Too Large" ;;
+ 451) response_message="451 Unavailable For Legal Reasons" ;;
+
+ # 5xx server errors
+ 500) response_message="500 Internal Server Error" ;;
+ 501) response_message="501 Not Implemented" ;;
+ 502) response_message="502 Bad Gateway" ;;
+ 503) response_message="503 Service Unavailable" ;;
+ 504) response_message="504 Gateway Timeout" ;;
+ 505) response_message="505 HTTP Version Not Supported" ;;
+ 506) response_message="506 Variant Also Negotiates" ;;
+ 507) response_message="507 Insufficient Storage" ;;
+ 508) response_message="508 Loop Detected" ;;
+ 510) response_message="510 Not Extended" ;;
+ 511) response_message="511 Network Authentication Required" ;;
+esac
+
+response_message="Server responded ${response_message}"
+case ${response_message:17:1} in
+ 2) notify "$response_message" ;;
+ 3) warn "$response_message" ;;
+ 4|5) error "$response_message" ;;
+esac
diff --git a/scripts/options b/scripts/options
new file mode 100644
index 0000000..a6aeb6d
--- /dev/null
+++ b/scripts/options
@@ -0,0 +1,129 @@
+# specify arbitrary curl arguments - passed directly to curl
+c() {
+ curl_args="$1"
+}
+
+# help
+h() {
+ echo "Usage: rsstube [OPTIONS] URL"
+ echo "Do 'man rsstube' to see the manual for rsstube."
+ exit
+}
+
+# non-network mode (do not download page)
+n() {
+ network="false"
+}
+
+# proxy
+p() {
+ proxy="-x $1 "
+}
+
+# quiet (use curl in silent mode)
+q() {
+ quiet="--silent "
+}
+
+# curl options for Tor (to avoid Cloudflare blocks)
+t() {
+ tor_header=""
+ while read line
+ do
+ tor_header="${tor_header}${line} "
+ done < conf/tor-browser-header
+ curl_args="${curl_args} ${tor_header}"
+}
+
+# update rsstube
+U() {
+ update="true"
+}
+
+# display version
+V() {
+ version=`cat docs/version`
+ echo "rsstube v${version}"
+ exit
+}
+
+global_config="/etc/rsstube/config"
+local_config="$HOME/.config/rsstube/config"
+config_file=""
+
+# local_config takes precedence over global_config, and only one (at most) will be used
+if [[ -f "$local_config" ]]
+then
+ config_file=${local_config}
+elif [[ -f "$global_config" ]]
+then
+ config_file=${global_config}
+fi
+
+# default assume we're not using Tor mode
+tor_mode=""
+
+if [[ -n "$config_file" ]]
+then
+ while read line
+ do
+ # cut whitespace from beginning of line
+ line=${line##*( )}
+
+ # ignore commented out lines
+ if [[ -n $line && ${line:0:1} != "#" ]]
+ then
+ # get just argument
+ cmd=${line%% *}
+
+ case $cmd in
+ -c|--curl-args) c "${line#* }" ;;
+ -h|--help|help) h ;;
+ -n|--non-network) n ;;
+ -p|--proxy) p "${line#* }" ;;
+ -q|--quiet) q ;;
+ -t|--tor-mode) tor_mode="true" ;;
+ -U|--update) U ;;
+ -V|--version) V ;;
+ *) error "Unsupported option."; exit ;;
+ esac
+ fi
+ done < "$config_file"
+fi
+
+# options from method call take precedence over config file options
+while [[ $# -gt 0 ]]
+do
+ cmd="${1}"
+ case $cmd in
+ -c|--curl-args) shift; c "${1}" ;;
+ -h|--help|help) h ;;
+ -n|--non-network) n ;;
+ -p|--proxy) shift; p "${1}" ;;
+ -q|--quiet) q ;;
+ -t|--tor-mode) tor_mode="true" ;;
+ -U|--update) U ;;
+ -V|--version) V ;;
+ *) url="${1}" ;;
+ esac
+
+ # finish with this command
+ shift
+done
+
+if [[ "${update}" == "true" ]]
+then
+ source scripts/update
+ exit
+fi
+
+if [[ -n "${tor_mode}" ]]
+then
+ t
+fi
+
+if [[ ! -n "${url}" ]]
+then
+ error "No URL supplied!"
+ exit
+fi
diff --git a/scripts/page-extractors/apple_podcasts b/scripts/page-extractors/apple_podcasts
new file mode 100644
index 0000000..19fc05a
--- /dev/null
+++ b/scripts/page-extractors/apple_podcasts
@@ -0,0 +1,6 @@
+rss=`echo "$content" | grep -o -P '(?<="feedUrl":").*?(?=")' | head -1`
+
+if [[ -n "$rss" ]]
+then
+ page_ext=$rss
+fi
diff --git a/scripts/page-extractors/castro_fm b/scripts/page-extractors/castro_fm
new file mode 100644
index 0000000..1fc550a
--- /dev/null
+++ b/scripts/page-extractors/castro_fm
@@ -0,0 +1 @@
+page_ext=`echo "$content" | grep -Pzo '(?<=![Subscribe to RSS]()
).*?(?=)' | head -1`
+
+if [[ -n "$username" ]]
+then
+ rss="https://www.chirbit.com/$username/rss"
+ page_ext=$rss
+fi
diff --git a/scripts/page-extractors/fyyd b/scripts/page-extractors/fyyd
new file mode 100644
index 0000000..2a212d5
--- /dev/null
+++ b/scripts/page-extractors/fyyd
@@ -0,0 +1 @@
+page_ext=`echo "$content" | grep -Po '(?<=)' | head -1 | tr '\0' '\n'`
diff --git a/scripts/page-extractors/google_podcasts b/scripts/page-extractors/google_podcasts
new file mode 100644
index 0000000..29d874b
--- /dev/null
+++ b/scripts/page-extractors/google_podcasts
@@ -0,0 +1 @@
+page_ext=`echo "$content" | grep -Po '(?<=data-feed=").*?(?=">)' | head -1 | tr '\0' '\n'`
diff --git a/scripts/page-extractors/invidious b/scripts/page-extractors/invidious
new file mode 100644
index 0000000..3a6ae95
--- /dev/null
+++ b/scripts/page-extractors/invidious
@@ -0,0 +1,11 @@
+determine_protocol
+get_domain
+instance="${protocol}${domain}"
+
+channel=`echo $content | grep -Pzo '(?<=
\n
"
+)
+
+nitter=(
+ ''
+)
+
+# note: we can identify PeerTube, but I don't know how to get the feed yet
+#peertube=(
+# ''
+#)
+
+pleroma=(
+ ''
+ 'Developers
\n- \n
- Documentation \n
- API \n
)' | head -1 | tr '\0' '\n'`
+channel="${channel##/}"
+
+if [[ -n $channel ]]
+then
+ page_ext="${instance}/feed/${channel}"
+fi
diff --git a/scripts/page-extractors/mastodon b/scripts/page-extractors/mastodon
new file mode 100644
index 0000000..3825d33
--- /dev/null
+++ b/scripts/page-extractors/mastodon
@@ -0,0 +1,26 @@
+# check for explicit feed link
+page_ext=`echo $content | grep -Po "(?<=)" | head -1`
+page_ext=${page_ext##*href=\'}
+
+# if explicit feed link not found get user URL and add .rss
+if [[ "$page_ext" == "" ]]
+then
+ determine_protocol
+ get_domain
+ username="${url##*//}"
+ username="${username#*/}"
+
+ if [[ "$username" == @* ]]
+ then
+ username="${username%%/*}"
+ elif [[ "$username" == users/* ]]
+ then
+ username="${username#*/}"
+ username="users/${username%%/*}"
+ fi
+
+ if [[ -n "$username" ]]
+ then
+ page_ext="${protocol}${domain}/${username}.rss"
+ fi
+fi
diff --git a/scripts/page-extractors/player_fm b/scripts/page-extractors/player_fm
new file mode 100644
index 0000000..9efa725
--- /dev/null
+++ b/scripts/page-extractors/player_fm
@@ -0,0 +1 @@
+page_ext=`echo "$content" | grep -Po '(?<=•Feed
)'`
diff --git a/scripts/page-extractors/pleroma b/scripts/page-extractors/pleroma
new file mode 100644
index 0000000..e7f665a
--- /dev/null
+++ b/scripts/page-extractors/pleroma
@@ -0,0 +1,25 @@
+determine_protocol
+get_domain
+
+# check for explicit feed link
+feed_link=`echo "$content" | grep -Po '(?<=)' | head -1 | tr '\0' '\n'`
+if [[ -n $feed_link ]]
+then
+ page_ext="${protocol}${domain}${feed_link}"
+fi
+
+# if explicit feed link not found get user URL and add /feed.atom
+if [[ "$page_ext" == "" ]]
+then
+ determine_protocol
+ get_domain
+ username="${url##*//}"
+ username="${username#*/}"
+
+ if [[ "$username" == users/* ]]
+ then
+ username="${username#*/}"
+ username="users/${username%%/*}"
+ page_ext="${protocol}${domain}/${username}/feed.atom"
+ fi
+fi
diff --git a/scripts/page-extractors/plume b/scripts/page-extractors/plume
new file mode 100644
index 0000000..91c1219
--- /dev/null
+++ b/scripts/page-extractors/plume
@@ -0,0 +1,19 @@
+page_ext=`echo "$content" | grep -Po "(?<=)" | head -1`
+
+determine_protocol
+get_domain
+
+if [[ ! -n $page_ext ]]
+then
+ # blog page
+ if [[ "$url" == *${domain}/~/* ]]
+ then
+ # get link to user page
+ user_link=`echo $content | grep -Po '(?<=)' | head -1`
+ if [[ -n $user_link ]]
+ then
+ # get Atom feed
+ page_ext="${user_link%%/}/atom.xml"
+ fi
+ fi
+fi
diff --git a/scripts/page-extractors/pocketcasts b/scripts/page-extractors/pocketcasts
new file mode 100644
index 0000000..b45fd1b
--- /dev/null
+++ b/scripts/page-extractors/pocketcasts
@@ -0,0 +1 @@
+page_ext=`echo "$content" | grep -Po '(?<=)' | head -1 | tr '\u' '\n'`
diff --git a/scripts/page-extractors/soundcloud b/scripts/page-extractors/soundcloud
new file mode 100644
index 0000000..481bfb1
--- /dev/null
+++ b/scripts/page-extractors/soundcloud
@@ -0,0 +1,7 @@
+users=`echo $content | grep -o -P '(?<=soundcloud:users:).*?(?=")' | head -1`
+
+if [[ -n "$users" ]]
+then
+ rss="https://feeds.soundcloud.com/users/soundcloud:users:"${users}"/sounds.rss"
+ page_ext=$rss
+fi
diff --git a/scripts/page-extractors/vimeo b/scripts/page-extractors/vimeo
new file mode 100644
index 0000000..b91f737
--- /dev/null
+++ b/scripts/page-extractors/vimeo
@@ -0,0 +1,20 @@
+username=`echo "$content" | grep -o -P '(?<="creator_id":).*?(?=,)' | head -1`
+
+# try multiple labels, in case some only work sometimes
+
+if [[ ! -n "$username" ]]
+then
+ username=`echo $content | grep -o -P '(?<="owner":{"id":).*?(?=,"display_name":")' | head -1`
+fi
+
+if [[ ! -n "$username" ]]
+then
+ username=`echo $content | grep -o -P '(?<="item":{"@id":"https://vimeo.com/user).*?(?=","name":)' | head -1`
+fi
+
+# return URL only if username was found
+if [[ -n "$username" ]]
+then
+ rss="https://vimeo.com/user$username/videos/rss"
+ page_ext=$rss
+fi
diff --git a/scripts/page-extractors/writefreely b/scripts/page-extractors/writefreely
new file mode 100644
index 0000000..f859ee0
--- /dev/null
+++ b/scripts/page-extractors/writefreely
@@ -0,0 +1,21 @@
+# check for explicit feed link
+page_ext=`echo "$content" | grep -Po '(?<=)' | head -1 | tr '\0' '\n'`
+page_ext=${page_ext##*href=\"}
+
+# if explicit feed link not found (article, not author page), truncate post URL to get author URL
+if [[ "$page_ext" == "" ]]
+then
+ # check for invalid link that cannot be shortened
+ # I don't think this is necessary, but just in case
+ if [[ "${url:0:8}" != "https://" && "${url:0:7}" != "http://" ]] && [[ "${url%%/}" == */* ]] || [[ "${url%%/}" == http*://*/* ]]
+ then
+ # truncate tailing slash (if applicable)
+ rss=${url%%/}
+
+ # truncate last item
+ rss=${rss%/*}
+
+ # add /feed/
+ page_ext=${rss}/feed/
+ fi
+fi
diff --git a/scripts/page-extractors/youtube b/scripts/page-extractors/youtube
new file mode 100644
index 0000000..10eb1ba
--- /dev/null
+++ b/scripts/page-extractors/youtube
@@ -0,0 +1,19 @@
+# if page is a channel page, use canonical link to determine channel id
+try1=`echo "$content" | grep -Po '(?<=)'`
+if [[ -n "$try1" ]]; then
+ channel_id="$try1"
+
+else
+ # else (e.g., video page) extract just channel id from metadata
+ try2=`echo "$content" | grep -Po '(?<="channelId":").+?(?=")' | head -1`
+ if [[ -n "$try2" ]]; then
+ channel_id="$try2"
+ fi
+fi
+
+# if channel id successfully extracted
+if [[ -n ${channel_id} ]]
+then
+ rss="https://www.youtube.com/feeds/videos.xml?channel_id=${channel_id}"
+ page_ext=$rss
+fi
diff --git a/scripts/update b/scripts/update
new file mode 100644
index 0000000..79b4330
--- /dev/null
+++ b/scripts/update
@@ -0,0 +1,12 @@
+# must be able to download a new version
+if [[ ! -O .. && $EUID -ne 0 ]]
+then
+ error "Please run as root or the owner of this directory."
+ exit
+fi
+
+# if it's the git version, just git pull
+if [[ -d .git ]]
+then
+ git pull
+fi
diff --git a/scripts/url-extractors/chirbit b/scripts/url-extractors/chirbit
new file mode 100644
index 0000000..e21e7ac
--- /dev/null
+++ b/scripts/url-extractors/chirbit
@@ -0,0 +1,16 @@
+# can only extract feed from URL if it's chirbit.com
+if [[ $url == *"chirbit.com/"* ]]
+then
+ # get username (cut off https://www.chirbit.com/ from front)
+ username=${url##*chirbit.com/}
+
+ # cut off any extra stuff from end
+ username=${username%%/*}
+
+ if [[ -n $username ]]
+ then
+ # prepend chirbit URL and append /rss to form feed
+ rss="https://www.chirbit.com/${username}/rss"
+ url_ext=$rss
+ fi
+fi
diff --git a/scripts/url-extractors/deviantart b/scripts/url-extractors/deviantart
new file mode 100644
index 0000000..22b3d7f
--- /dev/null
+++ b/scripts/url-extractors/deviantart
@@ -0,0 +1,11 @@
+# get username (cut off https://www.deviantart.com/ from front)
+username=${url##*deviantart.com/}
+
+# cut off any extra stuff from end
+username=${username%%/*}
+
+if [[ -n $username ]]
+then
+ rss="https://backend.deviantart.com/rss.xml?type=deviation&q=by%3A${username}+sort%3Atime+meta%3Aall"
+ url_ext=$rss
+fi
diff --git a/scripts/url-extractors/hugo b/scripts/url-extractors/hugo
new file mode 100644
index 0000000..d5f215e
--- /dev/null
+++ b/scripts/url-extractors/hugo
@@ -0,0 +1,7 @@
+determine_protocol
+get_domain
+
+if [[ -n "$domain" ]]
+then
+ url_ext="${protocol}${domain}/index.xml"
+fi
diff --git a/scripts/url-extractors/invidious b/scripts/url-extractors/invidious
new file mode 100644
index 0000000..902890b
--- /dev/null
+++ b/scripts/url-extractors/invidious
@@ -0,0 +1,39 @@
+# determine whether HTTP or HTTPS
+# Tor instances may not use HTTPS
+determine_protocol
+
+get_domain
+
+instance=${protocol}${domain}
+
+# get feed if able
+channel_id=""
+list=""
+if [[ $url == *"/channel/"* ]]
+then
+ channel_id=${url#*/channel/}
+# Invidious just redirects /user/ to /channel/ anyway
+elif [[ $url == *"/user/"* ]]
+then
+ channel_id=${url#*/user/}
+# playlist
+elif [[ $url == *"&list="* ]]
+then
+ list=${url#*&list=}
+ list=${list%%&*}
+fi
+
+# trim extra arguments
+channel_id=${channel_id%%\?*}
+channel_id=${channel_id%%\&*}
+
+if [[ -n $channel_id ]]
+then
+ channel_id=${channel_id%%/*}
+ rss="${instance}/feed/channel/${channel_id}"
+ url_ext=$rss
+elif [[ -n $list ]]
+then
+ rss="${instance}/feed/playlist/${list}"
+ url_ext=$rss
+fi
diff --git a/scripts/url-extractors/nitter b/scripts/url-extractors/nitter
new file mode 100644
index 0000000..8a4067d
--- /dev/null
+++ b/scripts/url-extractors/nitter
@@ -0,0 +1,16 @@
+determine_protocol
+get_domain
+
+instance=${protocol}${domain}
+
+# get username
+username=${url##*//}
+username=${username#*/}
+username=${username%%/*}
+
+# get feed if able
+if [[ -n $username ]]
+then
+ rss="${instance}/${username}/rss"
+ url_ext=$rss
+fi
diff --git a/scripts/url-extractors/plume b/scripts/url-extractors/plume
new file mode 100644
index 0000000..503b352
--- /dev/null
+++ b/scripts/url-extractors/plume
@@ -0,0 +1,11 @@
+determine_protocol
+get_domain
+
+# user page
+if [[ "$url" == *${domain}/@/* ]]
+then
+ # Atom feed should be at domain/@/username/atom.xml
+ username=${url##*${domain}/@/}
+ username=${username%%/*}
+ url_ext="${protocol}${domain}/@/${username}/atom.xml"
+fi
diff --git a/scripts/url-extractors/reddit b/scripts/url-extractors/reddit
new file mode 100644
index 0000000..8d490df
--- /dev/null
+++ b/scripts/url-extractors/reddit
@@ -0,0 +1,6 @@
+# cut off trailing slash if present (to prevent duplicates)
+url=${url%%/}
+
+# append /rss to form feed
+rss=${url}/.rss
+url_ext=$rss
diff --git a/scripts/url-extractors/tumblr b/scripts/url-extractors/tumblr
new file mode 100644
index 0000000..7e60ea2
--- /dev/null
+++ b/scripts/url-extractors/tumblr
@@ -0,0 +1,31 @@
+# not all Tumblr blogs support HTTPS
+determine_protocol
+
+get_domain
+
+# require subdomain
+if [[ $domain == "tumblr.com" ]]
+then
+ error "Must be on a subdomain, e.g., username.tumblr.com"
+ fail
+fi
+
+# cut off end of page thing that gets appended sometimes
+page=${url%%'#_=_'}
+
+# cut off trailing slash if present (to prevent duplicates)
+page=${page%%/}
+
+# strip domain
+page=${page##*tumblr.com}
+page=${page##/}
+
+# get rss
+if [[ -n $page ]]
+then
+ rss=${protocol}${domain}/${page}/rss
+else
+ rss=${protocol}${domain}/rss
+fi
+
+url_ext=$rss
diff --git a/scripts/url-extractors/vimeo b/scripts/url-extractors/vimeo
new file mode 100644
index 0000000..762e4ed
--- /dev/null
+++ b/scripts/url-extractors/vimeo
@@ -0,0 +1,16 @@
+# only able to extract from URL if URL is user page
+if [[ $url == *"vimeo.com/user"* ]]
+then
+ # get username (cut off https://vimeo.com/user from front)
+ username=${url##*vimeo.com/user}
+
+ # cut off any extra stuff from end
+ username=${username%%/*}
+
+ if [[ -n $username ]]
+ then
+ # prepend vimeo URL and append /rss to form feed
+ rss="https://vimeo.com/user${username}/videos/rss"
+ url_ext=$rss
+ fi
+fi
diff --git a/scripts/url-extractors/youtube b/scripts/url-extractors/youtube
new file mode 100644
index 0000000..a4e1df8
--- /dev/null
+++ b/scripts/url-extractors/youtube
@@ -0,0 +1,23 @@
+if [[ $url == *"youtube.com/channel/"* ]]
+then
+ channel_id=${url#*youtube.com/channel/}
+ channel_id=${channel_id%%/*}
+ channel_id=${channel_id%%\?*}
+ channel_id=${channel_id%%\&*}
+ rss="https://www.youtube.com/feeds/videos.xml?channel_id=$channel_id"
+ url_ext=$rss
+elif [[ $url == *"youtube.com/user/"* ]]
+then
+ username=${url#*youtube.com/user/}
+ username=${username%%/*}
+ username=${username%%\?*}
+ username=${username%%\&*}
+ rss="https://www.youtube.com/feeds/videos.xml?user=$username"
+ url_ext=$rss
+elif [[ $url == *"youtube.com/playlist/?list="* || $url == *"youtube.com/"*"&list="* ]]
+then
+ list=${url#*list=}
+ list=${list%%&*}
+ rss="https://www.youtube.com/feeds/videos.xml?playlist_id=$list"
+ url_ext=$rss
+fi
diff --git a/templates/config b/templates/config
new file mode 100644
index 0000000..3aad8ec
--- /dev/null
+++ b/templates/config
@@ -0,0 +1,15 @@
+## Sample config file
+## Uncomment lines by removing the # to use them.
+
+## Run in non-network mode (only try to determine feed from URL, do not download the page)
+# -n
+
+## Use SOCKS5 proxy on localhost:9150
+# -p socks5h://localhost:9150
+
+## Run in quiet mode (do not show output from curl)
+# -q
+
+## Pretend to be Tor Browser to evade Cloudflare blocks while using Tor
+## NOTE: This option does NOT itself set a Tor proxy. Please do so with -p.
+# -t