✏️ 编辑：charsetprober.cpython-39.opt-1.pyc
路径：
/usr/lib/python3.9/site-packages/chardet/__pycache__/charsetprober.cpython-39.opt-1.pyc
大小：3.35 KB · 修改：2022-03-25 13:26:00 · 权限：0644 · 只读（保存会失败）
← 返回目录
👁 查看
⬇ 下载
检测到二进制内容，直接保存可能损坏文件。请改用「上传替换」。
a ��=b� � @ s0 d dl Z d dlZddlmZ G dd� de�ZdS )� N� )� ProbingStatec @ sn e Zd ZdZddd�Zdd� Zedd� �Zd d � Zed d � �Z d d� Z e dd� �Z e dd� �Z e dd� �ZdS )� CharSetProbergffffff�?Nc C s d | _ || _t�t�| _d S �N)�_state� lang_filter�loggingZ getLogger�__name__�logger)�selfr � r �9/usr/lib/python3.9/site-packages/chardet/charsetprober.py�__init__' s zCharSetProber.__init__c C s t j| _d S r )r Z DETECTINGr �r r r r �reset, s zCharSetProber.resetc C s d S r r r r r r � charset_name/ s zCharSetProber.charset_namec C s d S r r )r �bufr r r �feed3 s zCharSetProber.feedc C s | j S r )r r r r r �state6 s zCharSetProber.statec C s dS )Ng r r r r r �get_confidence: s z CharSetProber.get_confidencec C s t �dd| �} | S )Ns ([ -])+� )�re�sub)r r r r �filter_high_byte_only= s z#CharSetProber.filter_high_byte_onlyc C s\ t � }t�d| �}|D ]@}|�|dd� � |dd� }|�� sL|dk rLd}|�|� q|S )u9 We define three types of bytes: alphabet: english alphabets [a-zA-Z] international: international characters [-ÿ] marker: everything else [^a-zA-Z-ÿ] The input buffer can be thought to contain a series of words delimited by markers. This function works to filter all words that contain at least one international character. All contiguous sequences of markers are replaced by a single space ascii character. This filter applies to all scripts which do not use English characters. s% [a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N���� �r )� bytearrayr �findall�extend�isalpha)r �filtered�wordsZwordZ last_charr r r �filter_international_wordsB s � z(CharSetProber.filter_international_wordsc C s� t � }d}d}tt| ��D ]n}| ||d � }|dkr<d}n |dkrHd}|dk r|�� s||kr�|s�|�| ||� � |�d� |d }q|s�|�| |d � � |S ) a� Returns a copy of ``buf`` that retains only the sequences of English alphabet and high byte characters that are not between <> characters. Also retains English alphabet and high byte characters immediately before occurrences of >. This filter can be applied to all scripts which contain both English characters and extended ASCII characters, but is currently only used by ``Latin1Prober``. Fr r � >� <Tr r N)r �range�lenr r )r r Zin_tag�prevZcurrZbuf_charr r r �filter_with_english_lettersg s" z)CharSetProber.filter_with_english_letters)N)r � __module__� __qualname__ZSHORTCUT_THRESHOLDr r �propertyr r r r � staticmethodr r" r( r r r r r # s $r )r r Zenumsr �objectr r r r r �<module> s
💾 保存
取消
保存为 UTF-8，换行统一为 LF