U g!@sdZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z ddl mZmZddlmZmZddlmZeeZd Zdddejfd d Zdejfd d ZddejfddZejdfddZejdfddZeddZeddZ ddZ!dS)z+Implements a simple wrapper around urlopen.N) lru_cache)parse)URLError)Requesturlopen)RegexMatchErrorMaxRetriesExceeded) regex_searchicCslddd}|r|||r8t|ts8tt|dd}|drXt||||d}ntdt ||d S) Nz Mozilla/5.0zen-US,en)z User-Agentzaccept-languageutf-8)encodinghttp)headersmethoddataz Invalid URL)timeout) update isinstancebytesjsondumpslower startswithr ValueErrorr)urlrr rrZ base_headersrequestr2/tmp/pip-unpacked-wheel-1a9f0fi6/pytube/request.py_execute_requests   rcCs(|dkr i}t|||d}|dS)zSend an http GET request. :param str url: The URL to perform the GET request for. :param dict extra_headers: Extra headers to add to the request :rtype: str :returns: UTF-8 encoded string of response N)r rr )rreaddecode)r extra_headersrresponserrrget(s r"cCsD|dkr i}|dkri}|ddit||||d}|dS)a:Send an http POST request. :param str url: The URL to perform the POST request for. :param dict extra_headers: Extra headers to add to the request :param dict data: The data to send on the POST request :rtype: str :returns: UTF-8 encoded string of response Nz Content-Typezapplication/json)r rrr )rrrr)rr rrr!rrrpost9sr#ccst|}d|j|j|jf}tt|j}d|d<|t|}d}t |||dD]}|V||7}qV| d}t d} |D]&} | | } | rt| dd } qd} | | kr| |d<|t|}t |||dEd H| d7} qd S) z{Read the response in sequence. :param str url: The URL to perform the GET request for. :rtype: Iterable[bytes] %s://%s/%s?rsq)r max_retries Segment-Count: (\d+)r N)rurlsplitschemenetlocpathdict parse_qslquery urlencodestreamsplitrecompilesearchintgroupr)rrr' split_urlbase_urlquerysZ segment_datachunk stream_infoZsegment_count_patternlinematch segment_countseq_numrrr seq_streamXs,      rCc csft}d}||krbt|t|d}d|d|}d}|d|krJtz"t|d|d|d|d}Wqtk r} zt| jtjrnW5d} ~ XYnt j j k rYnXq|d7}q8|tkr:z6t|dddd d|d} | d } t | }Wn4tttfk r8} zt| W5d} ~ XYnX|} | sJq|t| 7}| Vq:qdS) zyRead the response in chunks. :param str url: The URL to perform the GET request for. :rtype: Iterable[bytes] rr*zbytes=-z&range=GET)rrNlg]zContent-Length)default_range_sizeminrrrrreasonsocketrr clientIncompleteReadinfor8KeyError IndexErrorrloggererrorrlen) rrr' file_size downloadedZstop_posZ range_headertriesr!erespZ content_ranger=rrrr3sN          r3cCstt|dS)zFetch size in bytes of file at given URL :param str url: The URL to get the size of :returns: int: size in bytes of remote file content-length)r8head)rrrrfilesizesrYc Csd}t|}d|j|j|jf}tt|j}d|d<|t|}t |dd}| }|t |7}d}| d}d} |D].} zt t| | d}Wq~tk rYq~Xq~|dkrtd | d} | |kr| |d<|t|}|t t|d 7}| d7} q|S) zFetch size in bytes of file at given URL from sequential requests :param str url: The URL to get the size of :returns: int: size in bytes of remote file rr$r%rErr(r)r* seq_filesizerW)rr+r,r-r.r/r0r1r2rrrQr4r8r rrX) rZtotal_filesizer:r;r<r!Zresponse_valuerAr>Z segment_regexr?rBrrrr[s:      r[cCs"t|dd}dd|DS)zFetch headers returned http GET request. :param str url: The URL to perform the GET request for. :rtype: dict :returns: dictionary of lowercase headers HEADrZcSsi|]\}}||qSr)r).0kvrrr szhead..)rrLitems)rresponse_headersrrrrXs rX)"__doc__ http.clientr rloggingr5rI functoolsrurllibr urllib.errorrurllib.requestrrZpytube.exceptionsrrZpytube.helpersr getLogger__name__rOrF_GLOBAL_DEFAULT_TIMEOUTrr"r#rCr3rYr[rXrrrrs>      ! / @  3