U gX@sdZddlZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z mZddlmZmZddlmZmZeeZGdd d Zeed d d Zee ed d dZeee edddZeeedddZeed ddZeed ddZee ed ddZ edddZ!e e edddZ"e e#d d!d"Z$e e#d d#d$Z%e&d%d&d'Z'e&ed(d)d*Z(e&e#d(d+d,Z)e&e#d(d-d.Z*e&ed(d/d0Z+e&e#d(d1d2Z,e&e#d(d3d4Z-e&e#d(d5d6Z.d=e&e#d7d8d9Z/eed:d;d<Z0dS)>aX This module contains all logic necessary to decipher the signature. YouTube's strategy to restrict downloading videos is to send a ciphered version of the signature to the client, along with the decryption algorithm obfuscated in JavaScript. For the clients to play the videos, JavaScript must take the ciphered version, cycle it through a series of "transform functions," and then signs the media URL with the output. This module is responsible for (1) finding and extracting those "transform functions" (2) maps them to Python equivalents and (3) taking the ciphered signature and decoding it. N)chain)AnyCallableDictListOptionalTuple) ExtractErrorRegexMatchError)cache regex_search)find_object_from_startpointthrottling_array_splitc@sTeZdZedddZedddZeeddd Zeee ee fd d d Z d S)CipherjscCs|t||_td}||jd}|s6td|jd|ddd}t|||_ ddg|_ t ||_ t ||_d|_dS)Nz^\w+\Wr__init__Zcallerpatternz\w+\.(\w+)\(\w,(\d+)\)z\w+\[(\"\w+\")\]\(\w,(\d+)\))get_transform_plantransform_planrecompilesearchr rgroupget_transform_map transform_mapjs_func_patternsget_throttling_planthrottling_planget_throttling_function_arraythrottling_array calculated_n)selfrZ var_regexZ var_matchvarr&1/tmp/pip-unpacked-wheel-1a9f0fi6/pytube/cipher.pyrs      zCipher.__init__) initial_ncCs|jr |jStt|jD]}|j|dkr||j|<q|jD]}|jt|d}t|st|dtd|jdt |d|jt|d}t|dkr||q>t|dkr>|jt|d}|||q>d ||_|jS) z6Converts n to the correct value to prevent throttling.brz is not callable.zThrottling array:  ) r#rangelenr"r intcallableloggerdebugr join)r$r(istepZ curr_funcZ first_argZ second_argr&r&r' calculate_n0s&       zCipher.calculate_n)ciphered_signaturereturnc CsZt|}|jD]@}||\}}|j|||}tdd||||j|qd|S)aADecipher the signature. Taking the ciphered signature, applies the transform functions. :param str ciphered_signature: The ciphered signature sent in the ``player_config``. :rtype: str :returns: Decrypted signature required to download the media content. zOapplied transform function output: %s js_function: %s argument: %d function: %sr.)listrparse_functionrr3r4r5)r$r9 signaturejs_funcnameargumentr&r&r' get_signatureLs   zCipher.get_signaturer>r:cCsZtd|jD]8}t|}||}|r|\}}|t|fSqtddddS)aParse the Javascript transform function. Break a JavaScript transform function down into a two element ``tuple`` containing the function name and some integer-based argument. :param str js_func: The JavaScript version of the transform function. :rtype: tuple :returns: two element tuple containing the function name and an argument. **Example**: parse_function('DE.AJ(a,15)') ('AJ', 15) zparsing transform functionr<rrN) r3r4rrrrgroupsr1r )r$r>rregexZ parse_matchfn_nameZfn_argr&r&r'r<js     zCipher.parse_functionN) __name__ __module__ __qualname__strrr;r8rAr rr1r<r&r&r&r'rs r)rr:c Csrddddddddd d d d g }td |D]6}t|}||}|r*td ||dSq*tddddS)zExtract the name of the function responsible for computing the signature. :param str js: The contents of the base.js asset file. :rtype: str :returns: Function name from regex match zX\b[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*encodeURIComponent\s*\(\s*(?P[a-zA-Z0-9$]+)\(zg\b[a-zA-Z0-9]+\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*encodeURIComponent\s*\(\s*(?P[a-zA-Z0-9$]+)\(zi(?:\b|[^a-zA-Z0-9$])(?P[a-zA-Z0-9$]{2})\s*=\s*function\(\s*a\s*\)\s*{\s*a\s*=\s*a\.split\(\s*""\s*\)zS(?P[a-zA-Z0-9$]+)\s*=\s*function\(\s*a\s*\)\s*{\s*a\s*=\s*a\.split\(\s*""\s*\)z1(["\'])signature\1\s*,\s*(?P[a-zA-Z0-9$]+)\(z!\.sig\|\|(?P[a-zA-Z0-9$]+)\(zyt\.akamaized\.net/\)\s*\|\|\s*.*?\s*[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*(?:encodeURIComponent\s*\()?\s*(?P[a-zA-Z0-9$]+)\(z>\b[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*(?P[a-zA-Z0-9$]+)\(zM\b[a-zA-Z0-9]+\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*(?P[a-zA-Z0-9$]+)\(zH\bc\s*&&\s*a\.set\([^,]+\s*,\s*\([^)]*\)\s*\(\s*(?P[a-zA-Z0-9$]+)\(zS\bc\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*\([^)]*\)\s*\(\s*(?P[a-zA-Z0-9$]+)\(zfinding initial function name"finished regex search, matched: %sr+get_initial_function_namemultiplerN)r3r4rrrrr )rfunction_patternsrrDfunction_matchr&r&r'rKs0     rKcCs4tt|}d|}tdt||dddS)aExtract the "transform plan". The "transform plan" is the functions that the ciphered signature is cycled through to obtain the actual signature. :param str js: The contents of the base.js asset file. **Example**: ['DE.AJ(a,15)', 'DE.VR(a,3)', 'DE.AJ(a,51)', 'DE.VR(a,3)', 'DE.kT(a,51)', 'DE.kT(a,8)', 'DE.VR(a,3)', 'DE.kT(a,21)'] z.%s=function\(\w\){[a-z=\.\(\"\)]*;(.*);(?:.+)}zgetting transform planr+)r;)rescaperKr3r4r split)rr?rr&r&r'rs r)rr%r:cCsZdt|}tdtj|tjd}||}|sBtd|d|d dd d S) aExtract the "transform object". The "transform object" contains the function definitions referenced in the "transform plan". The ``var`` argument is the obfuscated variable name which contains these functions, for example, given the function call ``DE.AJ(a,15)`` returned by the transform plan, "DE" would be the var. :param str js: The contents of the base.js asset file. :param str var: The obfuscated variable name that stores an object with all functions that descrambles the signature. **Example**: >>> get_transform_object(js, 'DE') ['AJ:function(a){a.reverse()}', 'VR:function(a,b){a.splice(0,b)}', 'kT:function(a,b){var c=a[0];a[0]=a[b%a.length];a[b]=c}'] zvar %s={(.*?)};zgetting transform object)flagsget_transform_objectrr+r* z, ) rrPr3r4rDOTALLrr rreplacerQ)rr%rrDZtransform_matchr&r&r'rSs   rScCs<t||}i}|D]$}|dd\}}t|}|||<q|S)aWBuild a transform function lookup. Build a lookup table of obfuscated JavaScript function names to the Python equivalents. :param str js: The contents of the base.js asset file. :param str var: The obfuscated variable name that stores an object with all functions that descrambles the signature. :r+)rSrQ map_functions)rr%Ztransform_objectmapperobjr?functionfnr&r&r'rs  rc Csdg}td|D]}t|}||}|rtd|t|dkrZ|dS|d}|r|d}tdj t |dd|}|r|dd d }d d |D}|t |Sqt d d ddS)zExtract the name of the function that computes the throttling parameter. :param str js: The contents of the base.js asset file. :rtype: str :returns: The name of the function used to compute the throttling parameter. zha\.[a-zA-Z]\s*&&\s*\([a-z]\s*=\s*a\.get\("n"\)\)\s*&&\s*\([a-z]\s*=\s*([a-zA-Z0-9$]+)(\[\d+\])?\([a-z]\)z Finding throttling function namerJr+r,z[]zvar {nfunc}\s*=\s*(\[.+?\]);)Znfunc,cSsg|] }|qSr&)strip).0xr&r&r' %sz0get_throttling_function_name..get_throttling_function_namerLrN)r3r4rrrr0rCrr^formatrPrQr1r )rrMrrDrNidxarrayr&r&r'rbs6      rbcCsZtt|}d|}t|}||}t||dd}d|}| d|S)zExtract the raw code for the throttling function. :param str js: The contents of the base.js asset file. :rtype: str :returns: The name of the function used to compute the throttling parameter. z%s=function\(\w\)r+r*r.r) rrPrbrrr spanrQr5r)rr?Z pattern_startrDmatchZcode_lines_listZ joined_linesr&r&r'get_throttling_function_code-s    rhc Csjt|}d}t|}||}t||dd}t|}g}|D]}z|t|WqFWnt k rtYnX|dkr|dqF| dr| dr||ddqF| dr0dt fd t fd tfd tfd tfd tfdtfdtfdt ff } d} | D]&\} } t| |r|| d} q| r0qF||qFtt|D]} || dkrH||| <qH|S)zExtract the "c" array. :param str js: The contents of the base.js asset file. :returns: The array of various integers, arrays, and functions. z,c=\[r+nullN"rr[zS{for\(\w=\(\w%\w\.length\+\w\.length\)%\w\.length;\w--;\)\w\.unshift\(\w.pop\(\)\)}{\w\.reverse\(\)}z{\w\.push\(\w\)}z.;var\s\w=\w\[0\];\w\[0\]=\w\[\w\];\w\[\w\]=\w}z case\s\d+z1\w\.splice\(0,1,\w\.splice\(\w,1,\w\[0\]\)\[0\]\)z;\w\.splice\(\w,1\)}zL\w\.splice\(-\w\)\.reverse\(\)\.forEach\(function\(\w\){\w\.unshift\(\w\)}\)zBfor\(var \w=\w\.length;\w;\)\w\.push\(\w\.splice\(--\w,1\)\[0\]\)}FT)rhrrrr rfrappendr1 ValueError startswithendswiththrottling_unshiftthrottling_reversethrottling_pushthrottling_swapthrottling_cipher_functionthrottling_nested_splice js_splicethrottling_prependr/r0)rraw_codeZ array_startZ array_regexrgZ array_rawZ str_arrayZconverted_arrayelrYfoundrr\r6r&r&r'r!FsT         r!rc Cst|}d}t|}||}t||dd}d}t|}||}g} |D]D}|ddkr| |d|d|dfqV| |d|dfqV| S)aExtract the "throttling plan". The "throttling plan" is a list of tuples used for calling functions in the c array. The first element of the tuple is the index of the function to call, and any remaining elements of the tuple are arguments to pass to that function. :param str js: The contents of the base.js asset file. :returns: The full function code for computing the throttlign parameter. ztry{r+z(c\[(\d+)\]\(c\[(\d+)\](,c(\[(\d+)\]))?\)r.r)rhrrrr rffindallrl) rrxZtransform_startZ plan_regexrgZtransform_plan_rawZ step_startZ step_regexmatchesZtransform_stepsr&r&r'rs      rarr_cCs|dddS)a@Reverse elements in a list. This function is equivalent to: .. code-block:: javascript function(a, b) { a.reverse() } This method takes an unused ``b`` variable as their transform functions universally sent two arguments. **Example**: >>> reverse([1, 2, 3, 4]) [4, 3, 2, 1] Nrr&r~r&r&r'reversesrrr)cCs ||dS)zAdd/remove items to/from a list. This function is equivalent to: .. code-block:: javascript function(a, b) { a.splice(0, b) } **Example**: >>> splice([1, 2, 3, 4], 2) [1, 2] Nr&rr&r&r'splicesrcCs>|t|}tt||g|d||dg||ddS)zSwap positions at b modulus the list length. This function is equivalent to: .. code-block:: javascript function(a, b) { var c=a[0];a[0]=a[b%a.length];a[b]=c } **Example**: >>> swap([1, 2, 3, 4], 2) [3, 2, 1, 4] r+rN)r0r;r)rr)rr&r&r'swaps r)rcCs4|ddd}tt|D]}||||<qdS)zReverses the input list. Needs to do an in-place reversal so that the passed list gets changed. To accomplish this, we create a reversed copy, and then change each indvidual element. Nr)copyr/r0)rZ reverse_copyr6r&r&r'rqsrqdecCs||dS)zPushes an element onto a list.N)rlrr&r&r'rrsrrcCs|t|t|t|S)zPerform the modular function from the throttling array functions. In the javascript, the modular operation is as follows: e = (e % d.length + d.length) % d.length We simply translate this to python here. )r0rr&r&r'throttling_mod_funcsrcCsFt||}|| d|d| }||D]}||q2dS)zRotates the elements of the list to the right. In the javascript, the operation is as follows: for(e=(e%d.length+d.length)%d.length;e--;)d.unshift(d.pop()) N)rclearrl)rrnew_arrryr&r&r'rps  rpc Cstd}d}t|}|}t|D]V\}}||||||d|t|}|||||||<|d8}q$dS)aThis ciphers d with e to generate a new list. In the javascript, the operation is as follows: var h = [A-Za-z0-9-_], f = 96; // simplified from switch-case loop d.forEach( function(l,m,n){ this.push( n[m]=h[ (h.indexOf(l)-h.indexOf(this[m])+m-32+f--)%h.length ] ) }, e.split("") ) z@ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_` r+N)r;r enumerateindexr0rl) rrhfthisZ copied_listmlZ bracket_valr&r&r'rt s, rtcCs2t||}t||d|d}t|dd|ddS)aNested splice function in throttling js. In the javascript, the operation is as follows: function(d,e){ e=(e%d.length+d.length)%d.length; d.splice( 0, 1, d.splice( e, 1, d[0] )[0] ) } While testing, all this seemed to do is swap element 0 and e, but the actual process is preserved in case there was an edge case that was not considered. r+rN)rrv)rrZ inner_splicer&r&r'ru-s rucCsbt|}t||}|| d|d| }||D]}||q:t|}||ks^tdS)a; In the javascript, the operation is as follows: function(d,e){ e=(e%d.length+d.length)%d.length; d.splice(-e).reverse().forEach( function(f){ d.unshift(f) } ) } Effectively, this moves the last e elements of d to the beginning. N)r0rrrlAssertionError)rrZ start_lenrryZend_lenr&r&r'rwQs  rwcCs*t||}|d}|||d<|||<dS)z6Swap positions of the 0'th and e'th elements in-place.rN)r)rrrr&r&r'rsps  rs)rstartcGsz,|t|krt|}|dkr*t||}Wntk rDd}YnX|rZ|t||krft||}||||}|d|t||||d}||D]}||q|S)aImplementation of javascript's splice function. :param list arr: Array to splice :param int start: Index at which to start changing the array :param int delete_count: Number of elements to delete from the array :param *items: Items to add to the array Reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/splice # noqa:E501 rN)r0 TypeErrorr;rrl)rrZ delete_countitemsZdeleted_elementsrryr&r&r'rvxs   $ rvrBcCsNdtfdtfdtfdtff}|D]\}}t||r |Sq tddddS) zFor a given JavaScript transform function, return the Python equivalent. :param str js_func: The JavaScript version of the transform function. rkz{\w\.splice\(0,\w\)}z9{var\s\w=\w\[0\];\w\[0\]=\w\[\w\%\w.length\];\w\[\w\]=\w}zD{var\s\w=\w\[0\];\w\[0\]=\w\[\w\%\w.length\];\w\[\w\%\w.length\]=\w}rXrLrN)rrrrrr )r>rYrr\r&r&r'rXs   rX)N)1__doc__loggingr itertoolsrtypingrrrrrrZpytube.exceptionsr r Zpytube.helpersr r Z pytube.parserr r getLoggerrFr3rrIrKrrSrrbrhr!rrr1rrr;rqrrrrprtrurwrsrvrXr&r&r&r's<   o$ .B#   "$*