X-Git-Url: https://git.rapsys.eu/youtubedl/blobdiff_plain/ecaee9d66905db1d5836d396c705d18d6e5f1f72..a7fd25993be110e616bb9bf6d602949b54df1a7d:/README.txt?ds=sidebyside diff --git a/README.txt b/README.txt index 54b6137..6ba6d68 100644 --- a/README.txt +++ b/README.txt @@ -20,7 +20,7 @@ youtube-dl - download videos from youtube.com or other video platforms INSTALLATION -To install it right away for all UNIX users (Linux, OS X, etc.), type: +To install it right away for all UNIX users (Linux, macOS, etc.), type: sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl sudo chmod a+rx /usr/local/bin/youtube-dl @@ -41,7 +41,7 @@ You can also use pip: This command will update youtube-dl if you have already installed it. See the pypi page for more information. -OS X users can install youtube-dl with Homebrew: +macOS users can install youtube-dl with Homebrew: brew install youtube-dl @@ -116,8 +116,8 @@ OPTIONS Network Options: --proxy URL Use the specified HTTP/HTTPS/SOCKS proxy. - To enable experimental SOCKS proxy, specify - a proper scheme. For example + To enable SOCKS proxy, specify a proper + scheme. For example socks5://127.0.0.1:1080/. Pass in an empty string (--proxy "") for direct connection --socket-timeout SECONDS Time to wait before giving up, in seconds @@ -131,16 +131,18 @@ Geo Restriction: --geo-verification-proxy URL Use this proxy to verify the IP address for some geo-restricted sites. The default proxy specified by --proxy (or none, if the - options is not present) is used for the + option is not present) is used for the actual downloading. --geo-bypass Bypass geographic restriction via faking - X-Forwarded-For HTTP header (experimental) + X-Forwarded-For HTTP header --no-geo-bypass Do not bypass geographic restriction via faking X-Forwarded-For HTTP header - (experimental) --geo-bypass-country CODE Force bypass geographic restriction with explicitly provided two-letter ISO 3166-2 - country code (experimental) + country code + --geo-bypass-ip-block IP_BLOCK Force bypass geographic restriction with + explicitly provided IP block in CIDR + notation Video Selection: @@ -227,10 +229,15 @@ Download Options: size. By default, the buffer size is automatically resized from an initial value of SIZE. + --http-chunk-size SIZE Size of a chunk for chunk-based HTTP + downloading (e.g. 10485760 or 10M) (default + is disabled). May be useful for bypassing + bandwidth throttling imposed by a webserver + (experimental) --playlist-reverse Download playlist videos in reverse order --playlist-random Download playlist videos in random order --xattr-set-filesize Set file xattribute ytdl.filesize with - expected file size (experimental) + expected file size --hls-prefer-native Use the native HLS downloader instead of ffmpeg --hls-prefer-ffmpeg Use ffmpeg instead of the native HLS @@ -249,7 +256,9 @@ Download Options: Filesystem Options: -a, --batch-file FILE File containing URLs to download ('-' for - stdin) + stdin), one URL per line. Lines starting + with '#', ';' or ']' are considered as + comments and ignored. --id Use only video ID in file name -o, --output TEMPLATE Output filename template, see the "OUTPUT TEMPLATE" for all the info @@ -465,9 +474,9 @@ Post-processing Options: default; fix file if we can, warn otherwise) --prefer-avconv Prefer avconv over ffmpeg for running the - postprocessors (default) - --prefer-ffmpeg Prefer ffmpeg over avconv for running the postprocessors + --prefer-ffmpeg Prefer ffmpeg over avconv for running the + postprocessors (default) --ffmpeg-location PATH Location of the ffmpeg/avconv binary; either the path to the binary or its containing directory. @@ -484,7 +493,7 @@ CONFIGURATION You can configure youtube-dl by placing any supported command line -option to a configuration file. On Linux and OS X, the system wide +option to a configuration file. On Linux and macOS, the system wide configuration file is located at /etc/youtube-dl.conf and the user wide configuration file at ~/.config/youtube-dl/config. On Windows, the user wide configuration file locations are %APPDATA%\youtube-dl\config.txt or @@ -567,8 +576,8 @@ However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to python string formatting operations. For example, %(NAME)s or %(NAME)05d. To clarify, that is a percent symbol followed by a name in -parentheses, followed by a formatting operations. Allowed names along -with sequence type are: +parentheses, followed by formatting operations. Allowed names along with +sequence type are: - id (string): Video identifier - title (string): Video title @@ -585,6 +594,8 @@ with sequence type are: available - upload_date (string): Video upload date (YYYYMMDD) - uploader_id (string): Nickname or id of the video uploader +- channel (string): Full name of the channel the video is uploaded on +- channel_id (string): Id of the channel - location (string): Physical location where the video was filmed - duration (numeric): Length of the video in seconds - view_count (numeric): How many users have watched the video on the @@ -1138,7 +1149,7 @@ Use the --cookies option, for example In order to extract cookies from browser use any conforming browser extension for exporting cookies. For example, cookies.txt (for Chrome) -or Export Cookies (for Firefox). +or cookies.txt (for Firefox). Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either # HTTP Cookie File or @@ -1346,12 +1357,12 @@ yourextractor): 1. Fork this repository 2. Check out the source code with: - git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git + git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git 3. Start a new git branch with - cd youtube-dl - git checkout -b yourextractor + cd youtube-dl + git checkout -b yourextractor 4. Start with this simple template and save it to youtube_dl/extractor/yourextractor.py: @@ -1408,10 +1419,13 @@ yourextractor): methods and a detailed description of what your extractor should and may return. Add tests and code for as many as you want. 8. Make sure your code follows youtube-dl coding conventions and check - the code with flake8. Also make sure your code works under all - Python versions claimed supported by youtube-dl, namely 2.6, 2.7, - and 3.2+. -9. When the tests pass, add the new files and commit them and push the + the code with flake8: + + $ flake8 youtube_dl/extractor/yourextractor.py + +9. Make sure your code works under all Python versions claimed + supported by youtube-dl, namely 2.6, 2.7, and 3.2+. +10. When the tests pass, add the new files and commit them and push the result, like this: $ git add youtube_dl/extractor/extractors.py @@ -1419,7 +1433,7 @@ yourextractor): $ git commit -m '[yourextractor] Add new extractor' $ git push origin yourextractor -10. Finally, create a pull request. We'll then review and merge it. +11. Finally, create a pull request. We'll then review and merge it. In any case, thank you very much for your contributions! @@ -1548,9 +1562,31 @@ fallback scenario: This code will try to extract from meta first and if it fails it will try extracting og:title from a webpage. -Make regular expressions flexible +Regular expressions + +Don't capture groups you don't use -When using regular expressions try to write them fuzzy and flexible. +Capturing group must be an indication that it's used somewhere in the +code. Any group that is not used must be non capturing. + +Example + +Don't capture id attribute name here since you can't use it for anything +anyway. + +Correct: + + r'(?:id|ID)=(?P\d+)' + +Incorrect: + + r'(id|ID)=(?P\d+)' + +Make regular expressions relaxed and flexible + +When using regular expressions try to write them fuzzy, relaxed and +flexible, skipping insignificant parts that are more likely to change, +allowing both single and double quotes for quoted values and so on. Example @@ -1578,11 +1614,48 @@ The code definitely should not look like: r'(.*?)', webpage, 'title', group='title') +Long lines policy + +There is a soft limit to keep lines of code under 80 characters long. +This means it should be respected if possible and if it does not make +readability and code maintenance worse. + +For example, you should NEVER split long string literals like URLs or +some other often copied entities over multiple lines to fit this limit: + +Correct: + + 'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4' + +Incorrect: + + 'https://www.youtube.com/watch?v=FqZTN594JQw&list=' + 'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4' + Use safe conversion functions -Wrap all extracted numeric data into safe functions from utils: -int_or_none, float_or_none. Use them for string to number conversions as -well. +Wrap all extracted numeric data into safe functions from +youtube_dl/utils.py: int_or_none, float_or_none. Use them for string to +number conversions as well. + +Use url_or_none for safe URL processing. + +Use try_get for safe metadata extraction from parsed JSON. + +Explore youtube_dl/utils.py for more useful convenience functions. + +More examples + +Safely extract optional description from parsed JSON + + description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str) + +Safely extract more optional metadata + + video = try_get(response, lambda x: x['result']['video'][0], dict) or {} + description = video.get('summary') + duration = float_or_none(video.get('durationMs'), scale=1000) + view_count = int_or_none(video.get('views'))