,h6ddlZddlZddlmZGddeZy)N) ProbingStateczeZdZdZd dZdZedZdZedZ dZ e d Z e d Z e d Zy) CharSetProbergffffff?Nc\d|_||_tjt|_yN)_state lang_filterlogging getLogger__name__logger)selfr s W/var/www/html/Resume-Scraper/venv/lib/python3.12/site-packages/chardet/charsetprober.py__init__zCharSetProber.__init__'s# &''1 c.tj|_yr)r DETECTINGr rs rresetzCharSetProber.reset,s",, rcyrrs r charset_namezCharSetProber.charset_name/srcyrr)rbufs rfeedzCharSetProber.feed3s rc|jSr)r rs rstatezCharSetProber.state6s {{rcy)Ngrrs rget_confidencezCharSetProber.get_confidence:src4tjdd|}|S)Ns([-])+ )resub)rs rfilter_high_byte_onlyz#CharSetProber.filter_high_byte_only=sff&c2 rct}tjd|}|D]C}|j|dd|dd}|j s|dkrd}|j|E|S)u9 We define three types of bytes: alphabet: english alphabets [a-zA-Z] international: international characters [€-ÿ] marker: everything else [^a-zA-Z€-ÿ] The input buffer can be thought to contain a series of words delimited by markers. This function works to filter all words that contain at least one international character. All contiguous sequences of markers are replaced by a single space ascii character. This filter applies to all scripts which do not use English characters. s%[a-zA-Z]*[-]+[a-zA-Z]*[^a-zA-Z-]?Nr") bytearrayr#findallextendisalpha)rfilteredwordsword last_chars rfilter_international_wordsz(CharSetProber.filter_international_wordsBs|;  O  'D OOD"I & RS I$$&9w+> OOI & 'rc>t}d}d}tt|D]a}|||dz}|dk(rd}n|dk(rd}|dks |jr1||kDr'|s%|j ||||j d|dz}c|s|j ||d |S) a Returns a copy of ``buf`` that retains only the sequences of English alphabet and high byte characters that are not between <> characters. Also retains English alphabet and high byte characters immediately before occurrences of >. This filter can be applied to all scripts which contain both English characters and extended ASCII characters, but is currently only used by ``Latin1Prober``. Frr>rCs: nFnr