Gitmask Anonymous PR #40
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rysson
reviewed
Mar 5, 2019
| @@ -30,7 +30,7 @@ def __init__(self): | |||
| self.domains = ['fili.cc'] | |||
|
|
|||
| self.base_link = 'https://fili.cc' | |||
| self.url_transl = 'embed?type=%s&code=%s&code2=%s&salt=%s' | |||
| self.url_transl = 'embed?type=%s&code=%s&code2=%s&salt=%s&title=title&title2=title2' | |||
There was a problem hiding this comment.
Ta strona potrzebuje stałego napisu w postaci title=title&title2=title2?
| @@ -77,7 +77,7 @@ def normalize(title): | |||
| try: | |||
| try: return title.decode('ascii').encode("utf-8") | |||
| except: pass | |||
| return str(''.join(c for c in unicodedata.normalize('NFKD', unicode(title.decode('utf-8'))) if unicodedata.category(c) != 'Mn')).replace('ł','l') | |||
| return str(''.join(c for c in unicodedata.normalize('NFKD', unicode(title.decode('utf-8'))) if unicodedata.category(c) != 'Mn').replace(u"\u0142",'l')) | |||
There was a problem hiding this comment.
A to ciekawe jest. Wiem, że już było i poprawka jest drobna, ale przy okazji:
- Czy ta funkcja służy do wycinania ogonków i innych narodowych śmieci?
- Czy mamy pewność, że tylko małe litery będą? Małe
Łjest obsłużone (nie dekomponuje się, to nie akcent), ale wielkie już nie. To samo z ręcznym przejściem na 'ąć..' na 'ac'.. - Po co jest
unicode(title.decode('utf-8'))skoro samotitle.decode('utf-8')zwraca unicode?
Może zamieniać resztę dziwnych znaków na ?, np.
maketrans = lambda s1, s2: dict(zip(map(ord, s1), map(ord, s2)))
unicode_translate_table = maketrans(u'łŁ–—\u2044•„”«»', u'lL--/.""<>')
def normalize(title):
"""Convert UTF-8 title to ASCII as well as we can."""
if not isinstance(title, type(u'')):
title = title.decode('utf-8')
title = u''.join(c for c in unicodedata.normalize('NFKD', title) if unicodedata.category(c) != 'Mn')
return title.translate(unicode_translate_table).encode('ascii', 'replace')Dzięki czemu mamy konwersję (zawsze można rozszerzyć unicode_translate_table jakby co):
| UTF-8 | ASCII |
|---|---|
| Zażółć AŁĆ | Zazolc ALC |
| • Hit – „Česki Film «Arabela ½»” | . Hit - "Ceski Film <Arabela 1/2>" |
| µ-film | ?-film |
Działa w Python2 i Python3.
EDIT: Dodałem wykrywanie czy to już nie jest unicode, nie trzeba będzie dawać encode przed wywołaniem, jak jest np. w #41.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an anonymous PR submitted via Gitmask - https://www.gitmask.com