E gdZddlmZddlmZddlmZddlZddlZddl m Z e rddl m Z m Z dZdd ZddZddZddZddZdS)z6 :func:`~pandas.eval` source string parsing functions ) annotations)StringIO) iskeywordN) TYPE_CHECKING)HashableIteratordnamestrreturnc x|rt|s|SdtjDdddddddd d d fd |D}d |}|std|d|S)a= Create valid Python identifiers from any string. Check if name contains any special characters. If it contains any special characters, the special characters will be replaced by a special string and a prefix is added. Raises ------ SyntaxError If the returned name is not a Python valid identifier, raise an exception. This can happen if there is a hashtag in the name, as the tokenizer will than terminate and not find the backtick. But also for characters that fall out of the range of (U+0001..U+007F). cBi|]\}}|dtj|dS)_)tokentok_name).0chartokvals T/var/www/sysmax/venv/lib/python3.11/site-packages/pandas/core/computation/parsing.py z2create_valid_python_identifier..,sA''' D& +%.(+++'''r_QUESTIONMARK__EXCLAMATIONMARK_ _DOLLARSIGN_ _EUROSIGN_ _DEGREESIGN_ _SINGLEQUOTE_ _DOUBLEQUOTE_) ?!$u€°'"c<g|]}||S)get)rrspecial_characters_replacementss r z2create_valid_python_identifier..@s*UUU377dCCUUUrBACKTICK_QUOTED_STRING_zCould not convert 'z' to a valid Python identifier.) isidentifierrtokenizeEXACT_TOKEN_TYPESitemsupdatejoin SyntaxError)r r*s @rcreate_valid_python_identifierr4s  9T?? ''%7==??'''#$**!$     77UUUUPTUUU V VD +T + +D     WUUUUVVV Krtoktuple[int, str]c`|\}}|tkrtjt|fS||fS)a[ Clean up a column name if surrounded by backticks. Backtick quoted string are indicated by a certain tokval value. If a string is a backtick quoted token it will processed by :func:`_create_valid_python_identifier` so that the parser can find this string when the query is executed. In this case the tok will get the NAME tokval. Parameters ---------- tok : tuple of int, str ints correspond to the all caps constants in the tokenize module Returns ------- tok : Tuple[int, str] Either the input or token or the replacement values )BACKTICK_QUOTED_STRINGr.NAMEr4)r5toknumrs rclean_backtick_quoted_toksr;Is9(NFF '''}rrc td|d}t|d}t|S#t$r|cYSwxYw)a Function to emulate the cleaning of a backtick quoted name. The purpose for this function is to see what happens to the name of identifier if it goes to the process of being parsed a Python code inside a backtick quoted string and than being cleaned (removed of any special characters). Parameters ---------- name : hashable Name to be cleaned. Returns ------- name : hashable Returns the name after tokenizing and cleaning. Notes ----- For some cases, a name cannot be converted to a valid Python identifier. In that case :func:`tokenize_string` raises a SyntaxError. In that case, we just return the name unmodified. If this name was used in the query string (this makes the query call impossible) an error will be raised by :func:`tokenize_backtick_quoted_string` instead, which is not caught and propagates to the user level. `)tokenize_stringnextr4r3)r tokenizedrs rclean_column_namerBcs^:#KKKK00 i#-f555  s69 AAtoken_generatorIterator[tokenize.TokenInfo]source string_startintcZ|D]\}}}}}|dkr |d}nt|||fS)a Creates a token from a backtick quoted string. Moves the token_generator forwards till right after the next backtick. Parameters ---------- token_generator : Iterator[tokenize.TokenInfo] The generator that yields the tokens of the source string (Tuple[int, str]). The generator is at the first token after the backtick (`) source : str The Python source code string. string_start : int This is the start of backtick quoted string inside the source string. Returns ------- tok: Tuple[int, str] The token that represents the backtick quoted string. The integer is equal to BACKTICK_QUOTED_STRING (100). r=r>)r8)rCrErFrrstart string_ends rtokenize_backtick_quoted_stringrKsQ4#265!Q S==qJ E  "6,z*A#B BBrIterator[tuple[int, str]]c#Kt|j}tj|}|D]Y\}}}}}|dkrE t |||ddzV-#t $r}t d|d|d}~wwxYw||fVZdS)a$ Tokenize a Python source code string. Parameters ---------- source : str The Python source code string. Returns ------- tok_generator : Iterator[Tuple[int, str]] An iterator yielding all tokens with only toknum and tokval (Tuple[ing, str]). r=r>)rFzFailed to parse backticks in 'z'.N)rreadliner.generate_tokensrK Exceptionr3)rE line_readerrCr:rrIrerrs rr?r?s6""+K.{;;O(7 ! !#q! S== X5#V%(Q, X X X!"M6"M"M"MNNTWW X&.  ! !sA A=$A88A=)r r r r )r5r6r r6)r rr r)rCrDrEr rFrGr r6)rEr r rL)__doc__ __future__riorkeywordrrr.typingrcollections.abcrrr8r4r;rBrKr?r(rrrYs#"""""   0000f4""""JCCCCD!!!!!!r