Alexander/ripper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

added peertube support

Browse Source
This commit is contained in:
Alexander Bocken 2021-06-26 20:35:43 +02:00
parent 92ccd5a226
commit c0ae248e62
Signed by: Alexander
GPG Key ID: 1D237BE83F9B05E8
2 changed files with 22 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@ -1,15 +1,21 @@
# ripper # ripper
Checks YT Channels and Bitchute channels for new videos and downloads them quickly Checks YouTube, Bitchute, and Peertube channels for new videos and downloads them quickly.
This is considerably faster than giving `youtube-dl` the channel URIs directly. This is considerably faster than giving `youtube-dl` the channel URIs directly.
## YouTube
This script can use the YouTube Data API v3 if available, but has a fallback if either: This script can use the YouTube Data API v3 if available, but has a fallback if either:
- no API Key is given - no API Key is given
- the Data quota for the day has been exceeded - the Data quota for the day has been exceeded
## Bitchute
Bitchute can also be used with this script. Bitchute can also be used with this script.
This part of the script is quite generic and could easily be adjusted for other websites. This part of the script is quite generic and could easily be adjusted for other websites.
## Peertube
Peertube instances require the channel RSS feed URL to work.
See the documentation inside the `ripper` file for more information. See the documentation inside the `ripper` file for more information.
## Install ## Install
@ -22,4 +28,3 @@ After `git clone`ing this repo, run the helper `installer`
## Features to come ## Features to come
- Odysee support - Odysee support
- Peertube support -> could just be done via RSS feeds

15
ripper
View File

@ -105,6 +105,20 @@ for channel in $BC_CHANNELS; do
done done
grep 'bitchute' "$DLARCHIVE" | sed 's/bitchute /https:\/\/www\.bitchute\.com\/video\//' >> /tmp/alreadydownloaded$$ grep 'bitchute' "$DLARCHIVE" | sed 's/bitchute /https:\/\/www\.bitchute\.com\/video\//' >> /tmp/alreadydownloaded$$
##PEERTUBE
echo "Scanning Peertube instances..."
PT_CHANNELS="$(grep -Ev '(bichute|youtube)' $CHANNELSFILE | grep -v '^#')"
for channel in $PT_CHANNELS; do
printf 'PT channel: %s\n' "$channel"
channelxml="$(curl -s "$channel")"
todownload="$(echo "$channelxml" | xml2 | grep '^/rss/channel/item/link=' | sed 's|^/rss/channel/item/link=||' | sort -u | grep -vf "$BLACKLIST" | tee -ai /tmp/todownload$$)"
#might not be always correct for all peertube instances?
channelinstance="$(echo "$channelxml" | xml2 | grep '^/rss/channel/link=' | sed 's|^/rss/channel/link=||')"
#creates fake video links for other instances but shouldn't be an issue
grep -E 'peertube' "$DLARCHIVE" | sed "s|peertube |$channelinstance/videos/watch/|" | grep "$todownload" >> /tmp/alreadydownloaded$$
done
##DOWNLOAD VIDEOS FROM ACCUMULATED LINKS ##DOWNLOAD VIDEOS FROM ACCUMULATED LINKS
grep -vf /tmp/alreadydownloaded$$ /tmp/todownload$$ | sort -u > /tmp/new_videos$$ grep -vf /tmp/alreadydownloaded$$ /tmp/todownload$$ | sort -u > /tmp/new_videos$$
rm -f /tmp/alreadydownloaded$$ /tmp/todownload$$ rm -f /tmp/alreadydownloaded$$ /tmp/todownload$$
@ -121,6 +135,7 @@ if [ "$number" -gt 0 ]; then
notify-send "Channel Ripper" "Finished downloading" notify-send "Channel Ripper" "Finished downloading"
fi fi
if [ "$number" -eq 0 ]; then if [ "$number" -eq 0 ]; then
echo "No new videos" echo "No new videos"
fi fi