X-Git-Url: https://git.rapsys.eu/youtubedl/blobdiff_plain/022b4bc7e7b31fb2ec2bb8baefcb48654f87ab0b..5c3cc50ad59a38eb0b2c8690d3dfd6f784860327:/youtube-dl.1

diff --git a/youtube-dl.1 b/youtube-dl.1
index 9765054..cb8f218 100644
--- a/youtube-dl.1
+++ b/youtube-dl.1
@@ -116,7 +116,7 @@ Do not emit color codes in output
 .TP
 .B \-\-proxy \f[I]URL\f[]
 Use the specified HTTP/HTTPS/SOCKS proxy.
-To enable experimental SOCKS proxy, specify a proper scheme.
+To enable SOCKS proxy, specify a proper scheme.
 For example socks5://127.0.0.1:1080/.
 Pass in an empty string (\-\-proxy "") for direct connection
 .RS
@@ -145,26 +145,31 @@ Make all connections via IPv6
 .TP
 .B \-\-geo\-verification\-proxy \f[I]URL\f[]
 Use this proxy to verify the IP address for some geo\-restricted sites.
-The default proxy specified by \-\-proxy (or none, if the options is not
+The default proxy specified by \-\-proxy (or none, if the option is not
 present) is used for the actual downloading.
 .RS
 .RE
 .TP
 .B \-\-geo\-bypass
 Bypass geographic restriction via faking X\-Forwarded\-For HTTP header
-(experimental)
 .RS
 .RE
 .TP
 .B \-\-no\-geo\-bypass
 Do not bypass geographic restriction via faking X\-Forwarded\-For HTTP
-header (experimental)
+header
 .RS
 .RE
 .TP
 .B \-\-geo\-bypass\-country \f[I]CODE\f[]
 Force bypass geographic restriction with explicitly provided two\-letter
-ISO 3166\-2 country code (experimental)
+ISO 3166\-2 country code
+.RS
+.RE
+.TP
+.B \-\-geo\-bypass\-ip\-block \f[I]IP_BLOCK\f[]
+Force bypass geographic restriction with explicitly provided IP block in
+CIDR notation
 .RS
 .RE
 .SS Video Selection:
@@ -353,7 +358,7 @@ Download playlist videos in random order
 .RE
 .TP
 .B \-\-xattr\-set\-filesize
-Set file xattribute ytdl.filesize with expected file size (experimental)
+Set file xattribute ytdl.filesize with expected file size
 .RS
 .RE
 .TP
@@ -885,12 +890,12 @@ One of never (do nothing), warn (only emit a warning), detect_or_warn
 .RE
 .TP
 .B \-\-prefer\-avconv
-Prefer avconv over ffmpeg for running the postprocessors (default)
+Prefer avconv over ffmpeg for running the postprocessors
 .RS
 .RE
 .TP
 .B \-\-prefer\-ffmpeg
-Prefer ffmpeg over avconv for running the postprocessors
+Prefer ffmpeg over avconv for running the postprocessors (default)
 .RS
 .RE
 .TP
@@ -916,7 +921,7 @@ srt|ass|vtt|lrc)
 .PP
 You can configure youtube\-dl by placing any supported command line
 option to a configuration file.
-On Linux and OS X, the system wide configuration file is located at
+On Linux and macOS, the system wide configuration file is located at
 \f[C]/etc/youtube\-dl.conf\f[] and the user wide configuration file at
 \f[C]~/.config/youtube\-dl/config\f[].
 On Windows, the user wide configuration file locations are
@@ -1028,7 +1033,7 @@ formatting
 operations (https://docs.python.org/2/library/stdtypes.html#string-formatting).
 For example, \f[C]%(NAME)s\f[] or \f[C]%(NAME)05d\f[].
 To clarify, that is a percent symbol followed by a name in parentheses,
-followed by a formatting operations.
+followed by formatting operations.
 Allowed names along with sequence type are:
 .IP \[bu] 2
 \f[C]id\f[] (string): Video identifier
@@ -1059,6 +1064,11 @@ became available
 .IP \[bu] 2
 \f[C]uploader_id\f[] (string): Nickname or id of the video uploader
 .IP \[bu] 2
+\f[C]channel\f[] (string): Full name of the channel the video is
+uploaded on
+.IP \[bu] 2
+\f[C]channel_id\f[] (string): Id of the channel
+.IP \[bu] 2
 \f[C]location\f[] (string): Physical location where the video was filmed
 .IP \[bu] 2
 \f[C]duration\f[] (numeric): Length of the video in seconds
@@ -1748,8 +1758,8 @@ In order to extract cookies from browser use any conforming browser
 extension for exporting cookies.
 For example,
 cookies.txt (https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg)
-(for Chrome) or Export
-Cookies (https://addons.mozilla.org/en-US/firefox/addon/export-cookies/)
+(for Chrome) or
+cookies.txt (https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/)
 (for Firefox).
 .PP
 Note that the cookies file must be in Mozilla/Netscape format and the
@@ -2081,15 +2091,24 @@ Have a look at
 \f[C]youtube_dl/extractor/common.py\f[] (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py)
 for possible helper methods and a detailed description of what your
 extractor should and may
-return (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L74-L252).
+return (https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303).
 Add tests and code for as many as you want.
 .IP " 8." 4
 Make sure your code follows youtube\-dl coding conventions and check the
-code with flake8 (https://pypi.python.org/pypi/flake8).
-Also make sure your code works under all
-Python (https://www.python.org/) versions claimed supported by
-youtube\-dl, namely 2.6, 2.7, and 3.2+.
+code with
+flake8 (http://flake8.pycqa.org/en/latest/index.html#quickstart):
+.RS 4
+.IP
+.nf
+\f[C]
+\ $\ flake8\ youtube_dl/extractor/yourextractor.py
+\f[]
+.fi
+.RE
 .IP " 9." 4
+Make sure your code works under all Python (https://www.python.org/)
+versions claimed supported by youtube\-dl, namely 2.6, 2.7, and 3.2+.
+.IP "10." 4
 When the tests pass, add (https://git-scm.com/docs/git-add) the new
 files and commit (https://git-scm.com/docs/git-commit) them and
 push (https://git-scm.com/docs/git-push) the result, like this:
@@ -2097,14 +2116,14 @@ push (https://git-scm.com/docs/git-push) the result, like this:
 .IP
 .nf
 \f[C]
-\ $\ git\ add\ youtube_dl/extractor/extractors.py
-\ $\ git\ add\ youtube_dl/extractor/yourextractor.py
-\ $\ git\ commit\ \-m\ \[aq][yourextractor]\ Add\ new\ extractor\[aq]
-\ $\ git\ push\ origin\ yourextractor
+$\ git\ add\ youtube_dl/extractor/extractors.py
+$\ git\ add\ youtube_dl/extractor/yourextractor.py
+$\ git\ commit\ \-m\ \[aq][yourextractor]\ Add\ new\ extractor\[aq]
+$\ git\ push\ origin\ yourextractor
 \f[]
 .fi
 .RE
-.IP "10." 4
+.IP "11." 4
 Finally, create a pull
 request (https://help.github.com/articles/creating-a-pull-request).
 We\[aq]ll then review and merge it.
@@ -2134,7 +2153,7 @@ update at all.
 .PP
 For extraction to work youtube\-dl relies on metadata your extractor
 extracts and provides to youtube\-dl expressed by an information
-dictionary (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L75-L257)
+dictionary (https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303)
 or simply \f[I]info dict\f[].
 Only the following meta fields in the \f[I]info dict\f[] are considered
 mandatory for a successful extraction process by youtube\-dl:
@@ -2155,7 +2174,7 @@ extraction does not make any sense without and if any of them fail to be
 extracted then the extractor is considered completely broken.
 .PP
 Any
-field (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.py#L149-L257)
+field (https://github.com/rg3/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303)
 apart from the aforementioned ones are considered \f[B]optional\f[].
 That means that extraction should be \f[B]tolerant\f[] to situations
 when sources for these fields can potentially be unavailable (even if
@@ -2276,9 +2295,37 @@ title\ =\ meta.get(\[aq]title\[aq])\ or\ self._og_search_title(webpage)
 .PP
 This code will try to extract from \f[C]meta\f[] first and if it fails
 it will try extracting \f[C]og:title\f[] from a \f[C]webpage\f[].
-.SS Make regular expressions flexible
+.SS Regular expressions
+.SS Don\[aq]t capture groups you don\[aq]t use
 .PP
-When using regular expressions try to write them fuzzy and flexible.
+Capturing group must be an indication that it\[aq]s used somewhere in
+the code.
+Any group that is not used must be non capturing.
+.SS Example
+.PP
+Don\[aq]t capture id attribute name here since you can\[aq]t use it for
+anything anyway.
+.PP
+Correct:
+.IP
+.nf
+\f[C]
+r\[aq](?:id|ID)=(?P<id>\\d+)\[aq]
+\f[]
+.fi
+.PP
+Incorrect:
+.IP
+.nf
+\f[C]
+r\[aq](id|ID)=(?P<id>\\d+)\[aq]
+\f[]
+.fi
+.SS Make regular expressions relaxed and flexible
+.PP
+When using regular expressions try to write them fuzzy, relaxed and
+flexible, skipping insignificant parts that are more likely to change,
+allowing both single and double quotes for quoted values and so on.
 .SS Example
 .PP
 Say you need to extract \f[C]title\f[] from the following HTML code:
@@ -2321,11 +2368,64 @@ title\ =\ self._search_regex(
 \ \ \ \ webpage,\ \[aq]title\[aq],\ group=\[aq]title\[aq])
 \f[]
 .fi
+.SS Long lines policy
+.PP
+There is a soft limit to keep lines of code under 80 characters long.
+This means it should be respected if possible and if it does not make
+readability and code maintenance worse.
+.PP
+For example, you should \f[B]never\f[] split long string literals like
+URLs or some other often copied entities over multiple lines to fit this
+limit:
+.PP
+Correct:
+.IP
+.nf
+\f[C]
+\[aq]https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4\[aq]
+\f[]
+.fi
+.PP
+Incorrect:
+.IP
+.nf
+\f[C]
+\[aq]https://www.youtube.com/watch?v=FqZTN594JQw&list=\[aq]
+\[aq]PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4\[aq]
+\f[]
+.fi
 .SS Use safe conversion functions
 .PP
-Wrap all extracted numeric data into safe functions from \f[C]utils\f[]:
+Wrap all extracted numeric data into safe functions from
+\f[C]youtube_dl/utils.py\f[] (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py):
 \f[C]int_or_none\f[], \f[C]float_or_none\f[].
 Use them for string to number conversions as well.
+.PP
+Use \f[C]url_or_none\f[] for safe URL processing.
+.PP
+Use \f[C]try_get\f[] for safe metadata extraction from parsed JSON.
+.PP
+Explore
+\f[C]youtube_dl/utils.py\f[] (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/utils.py)
+for more useful convenience functions.
+.SS More examples
+.SS Safely extract optional description from parsed JSON
+.IP
+.nf
+\f[C]
+description\ =\ try_get(response,\ lambda\ x:\ x[\[aq]result\[aq]][\[aq]video\[aq]][0][\[aq]summary\[aq]],\ compat_str)
+\f[]
+.fi
+.SS Safely extract more optional metadata
+.IP
+.nf
+\f[C]
+video\ =\ try_get(response,\ lambda\ x:\ x[\[aq]result\[aq]][\[aq]video\[aq]][0],\ dict)\ or\ {}
+description\ =\ video.get(\[aq]summary\[aq])
+duration\ =\ float_or_none(video.get(\[aq]durationMs\[aq]),\ scale=1000)
+view_count\ =\ int_or_none(video.get(\[aq]views\[aq]))
+\f[]
+.fi
 .SH EMBEDDING YOUTUBE\-DL
 .PP
 youtube\-dl makes the best effort to be a good command\-line program,