Sitelet https://github.com/python/cpython/issues/84508
Skip to content

Add tools for generating mappings_XX.h #84508

Closed
#93272
Closed
Add tools for generating mappings_XX.h#84508
#93272
Assignees
corona10
Labels
3.12only security fixes

Description

@corona10
corona10
opened on Apr 19, 2020
BPO 40328
Nosy @hyeshik, @vstinner, @methane, @serhiy-storchaka, @corona10
PRs
  • bpo-40328: Add tool for generating cjk mapping headers #19602
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-04-19.13:13:59.400>
labels = ['3.9']
title = 'Add tools for generating mappings_XX.h'
updated_at = <Date 2020-05-25.04:14:49.537>
user = 'https://github.com/corona10'

    bugs.python.org fields:

    activity = <Date 2020-05-25.04:14:49.537>
actor = 'corona10'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Demos and Tools']
creation = <Date 2020-04-19.13:13:59.400>
creator = 'corona10'
dependencies = []
files = []
hgrepos = []
issue_num = 40328
keywords = ['patch']
message_count = 4.0
messages = ['366777', '367682', '369844', '369845']
nosy_count = 5.0
nosy_names = ['hyeshik.chang', 'vstinner', 'methane', 'serhiy.storchaka', 'corona10']
pr_nums = ['19602']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue40328'
versions = ['Python 3.9']

    Activity

    1. corona10 commented on Apr 19, 2020

      @corona10
      corona10
      on Apr 19, 2020
      MemberAuthor

      AFAIK, there are no tools for generating mappings_XX.h under Modules/cjkcodecs.

      It would cause a maintainable issue,
      fortunately, I got the old source from https://github.com/BackupTheBerlios/cjkpython.
      And I success to port into Python3 version and successfully reproduce to generating header file except for mappings_hk.h mappings_tw.h.

      The patch will include a generator and Unicode mapping file.
      I am not the codec expert but hopefully it will help us.

    2. serhiy-storchaka
      added
      3.9 (EOL)end of life
      on Apr 19, 2020

    3. corona10 commented on Apr 29, 2020

      @corona10
      corona10
      on Apr 29, 2020
      MemberAuthor

      New changeset 113feb3 by Dong-hee Na in branch 'master':
      bpo-40328: Add tool for generating cjk mapping headers (GH-19602)
      113feb3

    4. methane commented on May 25, 2020

      @methane
      methane
      on May 25, 2020
      Member

      Would you close this issue?

    5. corona10 commented on May 25, 2020

      @corona10
      corona10
      on May 25, 2020
      MemberAuthor

      Would you close this issue?

      mappings_hk.h mappings_tw.h are still left :)

    6. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    7. sorcio commented on May 4, 2022

      @sorcio
      sorcio
      on May 4, 2022
      Contributor

      Hey @corona10, I happened upon this for other reasons. I found that Python's big5hkscs codec does not support the same character set that modern web browsers support.

      There has been at least one version of HKSCS (2008) that updated the Big5-encoded character set, and one more recent version (2016) which only added characters to Unicode (with no Big5 mapping), but might adjust some of the existing mappings. I still need to understand if this might break compatibility.

      Is it okay if I look into this and try to make a PR later? I've already done something for mappings_hk but need to look into the compatibility, so it might take a second.

    8. corona10 commented on May 11, 2022

      @corona10
      corona10
      on May 11, 2022
      MemberAuthor

      @sorcio Oh thanks, PR will be welcomed :)

    9. corona10
      self-assigned this
      on May 11, 2022
    10. sorcio
      added a commit that references this issue on May 26, 2022

      pythongh-84508: tool to generate cjk traditional chinese mappings

      1fb94f7
    11. corona10
      added 3 commits that reference this issue on May 28, 2022

      gh-84508: Add mapping files for Korean and Japanese. (gh-93309)

      dec1e93

      Revert "gh-84508: Add mapping files for Korean and Japanese. (gh-93309)"

      b81a299

      Revert "gh-84508: Add mapping files for Korean and Japanese. (gh-93309)…

      749dc4b
    12. corona10
      added
      3.12only security fixes
      and removed
      3.9 (EOL)end of life
      on May 29, 2022
    13. corona10
      closed this as completedin #93272on Jun 11, 2022
    14. corona10
      added a commit that references this issue on Jun 11, 2022

      gh-84508: tool to generate cjk traditional chinese mappings (gh-93272)

      733e15f
    15. serhiy-storchaka
      added a commit that references this issue on Jun 26, 2022

      Revert 30935 defer bpo45162 to 312 (#94285)

      1ffde3e
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.12only security fixes

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions