Roadmap v4.* release

[kows]

Rather a question since I don't know how I would have to reach out.

For one of our clients I'm looking to upgrade U8 towards U13.
They make use of BoboFacets so I kind of need to switch towards this version.
It has only been released as pre-release (almost a year), can it be considered stable by now?

Any plans on backtracking new changes of the v3.* version towards the v4.*?
Main change I'm looking towards is the new handling of corrupt indexes.

[Shazwazza]

The roadmap doc is here and i think is still relevant https://shazwazza.github.io/Examine/developerguides/roadmap.html

But that said, those 4 items are optional and could certainly come in later versions. There are PRs for these but are either incomplete or need reviving

• V4.0 Change C# version from 9 to 12. Remove .NET Standard targets #360
• V4.0 Facet Drill Down and Drill Sideways support #364
• V4.0 Bugfix Index Float field as a Single #365
• V4.0 Filter Support #366
• V4.0 Geo Spatial Search Support #367

Without feedback from the community, it is difficult to know if v4 is 'stable'. There's been no reported issues apart from one:

• Field $facets was not indexed with SortedSetDocValues #388

If you have feedback on v4, it would be great to get some of it listed here (good or bad, etc...)

I will certainly merge v3 up to v4 :) There's some active work/fixes in v3 that should be ready to be shipped soon and then I can merge up to v4 and get another beta of 4 out.

[nzdev]

Happy to discuss a revival. As far as I'm aware, all those prs are complete / pending review.

[rsnpettrov]

@Shazwazza I won't open a new issue just for that question.
I see that 4.0.0-beta.1 was released on 10/27/2023. It's been a long time since 4 has stayed in beta.
What do you think it will get out of beta and officially released?
We are particularly looking into the facets implementation of the new version.

[Shazwazza]

Hi all, sorry for the radio silence here.

Several people have been using 4 beta in the wild with great success but there are still some outstanding PRs/Issues to resolve. I'm also awaiting a few community members feedback, namely @callumbwhyte ;) to provide some insights into all the changes and what might be made simpler.

Then - there's a gigantic amount of work being done in Lucene.Net right now to get it to a point where we can ship an RTM v4. There's quite a few breaking changes in the current beta so I'm hesitant to ship this now and then have to ship another major to cater for the breaking changes in the Lucene RTM release. Then - there's the Umbraco RFC for search which needs further details since potentially one of the requirements will be facets and I need to ensure that this all aligns with those requirements.

Feel free to use v4 in prod, many people are doing just that.

[kows]

Got similar error using this version for Internal and External index on the CM on a CM-CD setup (Linux) on acceptance (not live yet).
LuceneDirectoryFactory = SyncedTempFileSystemDirectoryFactory
MainDomLock = FileSystemMainDomLock
MainDomKeyDiscriminator not set
No deployment slots used.
Not sure if the recovery added in v3 covers this case.

System.IO.FileNotFoundException: Could not find file '/home/site/wwwroot/umbraco/Data/TEMP/ExamineIndexes/InternalIndex/_gg.cfs'.
File name: '/home/site/wwwroot/umbraco/Data/TEMP/ExamineIndexes/InternalIndex/_gg.cfs'
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at Lucene.Net.Store.MMapDirectory.OpenInput(String name, IOContext context)
   at Lucene.Net.Replicator.IndexRevision.Open(String source, String fileName)
   at Lucene.Net.Replicator.LocalReplicator.ObtainFile(String sessionId, String source, String fileName)
   at Lucene.Net.Replicator.ReplicationClient.DoUpdate()
   at Lucene.Net.Replicator.ReplicationClient.UpdateNow()
   at Examine.Lucene.ExamineReplicator.ReplicateIndex()
   at Examine.Lucene.Directories.SyncedFileSystemDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Directories.DirectoryFactoryBase.<>c__DisplayClass2_0.<Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory>b__0(String s)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Examine.Lucene.Directories.DirectoryFactoryBase.Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Umbraco.Cms.Infrastructure.Examine.ConfigurationEnabledDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Directories.DirectoryFactoryBase.<>c__DisplayClass2_0.<Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory>b__0(String s)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Examine.Lucene.Directories.DirectoryFactoryBase.Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass2_0.<.ctor>b__1()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
   at System.Lazy`1.CreateValue()
   at Examine.Lucene.Providers.LuceneIndex.GetLuceneDirectory()
   at Examine.Lucene.Providers.LuceneIndex.IndexReady()
   at Examine.Lucene.Providers.LuceneIndex.PerformIndexItemsInternal(IEnumerable`1 valueSets, CancellationToken cancellationToken)
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass57_0.<PerformIndexItems>b__0()
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass89_0.<QueueTask>b__0(Task x)

System.IO.FileNotFoundException: Could not find file '/home/site/wwwroot/umbraco/Data/TEMP/ExamineIndexes/ExternalIndex/_c3.cfs'.
File name: '/home/site/wwwroot/umbraco/Data/TEMP/ExamineIndexes/ExternalIndex/_c3.cfs'
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at Lucene.Net.Store.MMapDirectory.OpenInput(String name, IOContext context)
   at Lucene.Net.Replicator.IndexRevision.Open(String source, String fileName)
   at Lucene.Net.Replicator.LocalReplicator.ObtainFile(String sessionId, String source, String fileName)
   at Lucene.Net.Replicator.ReplicationClient.DoUpdate()
   at Lucene.Net.Replicator.ReplicationClient.UpdateNow()
   at Examine.Lucene.ExamineReplicator.ReplicateIndex()
   at Examine.Lucene.Directories.SyncedFileSystemDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Directories.DirectoryFactoryBase.<>c__DisplayClass2_0.<Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory>b__0(String s)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Examine.Lucene.Directories.DirectoryFactoryBase.Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Umbraco.Cms.Infrastructure.Examine.ConfigurationEnabledDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Directories.DirectoryFactoryBase.<>c__DisplayClass2_0.<Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory>b__0(String s)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Examine.Lucene.Directories.DirectoryFactoryBase.Examine.Lucene.Directories.IDirectoryFactory.CreateDirectory(LuceneIndex luceneIndex, Boolean forceUnlock)
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass2_0.<.ctor>b__1()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
   at System.Lazy`1.CreateValue()
   at Examine.Lucene.Providers.LuceneIndex.GetLuceneDirectory()
   at Examine.Lucene.Providers.LuceneIndex.IndexReady()
   at Examine.Lucene.Providers.LuceneIndex.PerformIndexItemsInternal(IEnumerable`1 valueSets, CancellationToken cancellationToken)
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass57_0.<PerformIndexItems>b__0()
   at Examine.Lucene.Providers.LuceneIndex.<>c__DisplayClass89_0.<QueueTask>b__0(Task x)

[rsnpettrov]

Hey, I had the same error in Umbraco Cloud the other day, but the solution runs Examine 3.0.1 so it is unrelated to the version.