Skip to content

ICU crash on sorting when one codepoint is missing in collation #130

@KenZook

Description

@KenZook

Describe the bug

In Flex 9.0.8 (LT-20194) YurutiT project, Flex will crash when clicking the Choose Texts button. I discovered it was comparing "Yurutí Example Sentences" and "YURUTI KINSHIP TERMS" when it failed. The last i in the first word is NFD 69 301. This particular code point was not in the original collation rule

& i < ĩ <<< Ĩ << ĩ́ <<< Ĩ́

So I added it

& i < ĩ <<< Ĩ << í << ĩ́ <<< Ĩ́

and then the crash went away. So apparently ICU has some problem with this particular code point when it is not defined in the collation causing a crash. We are using an old ICU version. Perhaps the latest version would not have this crash. I would like to see us using a current version of ICU in Flex.

To Reproduce

Steps to reproduce the behavior:

  1. Restore Yuruti
  2. Go to Texts and Words
  3. Click the Choose Texts button in toolbar.
    It crashes with a windows error. Event viewer has:
    Application: FieldWorks.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an unhandled exception.
    Exception Info: System.AccessViolationException
    at Icu.NativeMethods.ucol_strcoll(SafeRuleBasedCollatorHandle, System.String, Int32, System.String, Int32)
    at Icu.Collation.RuleBasedCollator.Compare(System.String, System.String)
    at SIL.WritingSystems.IcuRulesCollator.Compare(System.String, System.String)
    at System.Collections.Generic.ArraySortHelper1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].InsertionSort(System._Canon[], Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>)
    at System.Collections.Generic.ArraySortHelper1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].IntroSort(System._Canon[], Int32, Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>)
    at System.Collections.Generic.ArraySortHelper1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].IntroSort(System._Canon[], Int32, Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>)
    at System.Collections.Generic.ArraySortHelper1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].IntrospectiveSort(System._Canon[], Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>)
    at System.Collections.Generic.ArraySortHelper1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Sort(System._Canon[], Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>)
    at System.Array.Sort[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](System._Canon[], Int32, Int32, System.Collections.Generic.IComparer1<System._Canon>) at System.Collections.Generic.List1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Sort(System.Comparison`1<System.__Canon>)
    at SIL.FieldWorks.IText.TextsTriStateTreeView.LoadTextsByGenreAndWithoutGenre()
    at SIL.FieldWorks.IText.TextsTriStateTreeView.LoadGeneralTexts()

(Ideally push a minimal solution that demonstrates the problem to https://gist.github.com)
Download YurutiT.zip from LT-20194.

Expected behavior

It should bring up the Choose Texts dialog

Screenshots

image

I was able to catch it in dnSpy 6.1.4.
image

image

Environment

  • OS: Windows 8.1
  • Exact version of icu.net 2.5.4+Branch.master.Sha.aa2e04611b4...
  • .NET Framework/Core version ??

Additional context

This is very tricky to get it to fail. Sometimes it works, but usually not.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions