lock: Add howdy support

[bonnjalal]

Add howdy (IR face recognition) support to the lock.

If the user computer support IR emitters he can use howdy ( Windows Hello™ style authentication for Linux )

• Triggered by pressing enter on an empty buffer

[PixelKhaos]

I'll test this, but at a glance, did you integrate it alongside fprint correctly, and not replace it? I assume you have access to both hardware?

And rather than empty password, adding an actual butto to toggle between howdy or fprint might be a nicer approach

[bonnjalal]

I'll test this, but at a glance, did you integrate it alongside fprint correctly, and not replace it? I assume you have access to both hardware?

And rather than empty password, adding an actual butto to toggle between howdy or fprint might be a nicer approach

Yes I integrated it alongside fprint, I didn't touch the fprint logic but it will be good if someone test it with fprint and make sure that the PR does not cause any problem, unfortunately I don't have a hardware support fprint right now.

Why go with the empty password approach because when the screen locked it automatically start howdy for recognition and all the 3 attempts wasted or the computer unlocked if you are in front of your computer, so it's more convenient to press some key when you ready to scan your face.

And I think right now you can enable/disable fprint and howdy from caelestia config if I'm not wrong.

[PixelKhaos]

Why go with the empty password approach because when the screen locked it automatically start howdy for recognition and all the 3 attempts wasted or the computer unlocked if you are in front of your computer, so it's more convenient to press some key when you ready to scan your face.

Oh I mean, instead of starting howdy's attempts after pressing enter, that it would do so with an actual UI element in the lockscreen, that way its visually clear to the user as well. With fprint there's several indicators like the fingerprint icon and the hardware itself lighting up.

[PixelKhaos]

Quick update, quickshell crashes on authenticating with Howdy, my normal fingerprint auth still works fine, so does password, but blank password to engage Howdy does light up the camera but yeah, crash before unlocking leaving one on the Hyprland "whoops" screen.

I'll try debugging it and look at the logs to see what exactly may be the cause
Though it could have been something with the no_confirmation config option for Howdy, but no dice.

[bonnjalal]

Why go with the empty password approach because when the screen locked it automatically start howdy for recognition and all the 3 attempts wasted or the computer unlocked if you are in front of your computer, so it's more convenient to press some key when you ready to scan your face.

Oh I mean, instead of starting howdy's attempts after pressing enter, that it would do so with an actual UI element in the lockscreen, that way its visually clear to the user as well. With fprint there's several indicators like the fingerprint icon and the hardware itself lighting up.

Hi @PixelKhaos, thanks for the feedback!

I think I understand what you mean. Let me clarify my thinking first:

• In my testing, Howdy doesn't scan automatically when the screen locks. My PR was intended to add a new trigger for it, not replace an existing automatic one.

• I completely agree that visual feedback is essential. My PR does include this: when you press Enter on the empty buffer, the lock icon immediately changes to an eye icon to show that Howdy is actively scanning. This was inspired by the fprint behavior. (We can make that more obvious by adding a small message that tell the user click enter if you want to use howdy "If howdy exist" )

• Why "Press Enter"? I chose the "press Enter" method because it's a very fast, keyboard-friendly approach that I've seen in other popular lockers like hyprlock.

• Your "Clickable UI" Idea: I understand you're suggesting a clickable icon to trigger the scan, which is a valid approach. My main concern was how to handle the UI for users who have both fprint and Howdy enabled.

  • If we have a clickable icon (I suggest to be the lock icon itself when clicked it cycle between the auth methods that exist in the system), but the user have to click it multiple times to cycle between "Password," "Fprint," and "Howdy" if he have all this three on his system .

My current logic :

• Type + Enter = Always checks the password.
• Empty + Enter = Tries howdy (it attempts Howdy first) (I can make it if howdy not available or fails, it tries fprint if exist).

What are your thoughts on this? Let me know what you think the best next step is, or if you have any other suggestions.

[PixelKhaos]

What are your thoughts on this? Let me know what you think the best next step is, or if you have any other suggestions.

Oh yeah, using enter to skip straight to Howdy is absolutely fine, a button or some UI element would moreso be in addition rather than instead of. I dont think password should be in the same cycle though, that should always be present, but icon to switch between fprint and howdy would fit nicely.

I still have some issues debugging my issues with this, since it seems an older issue I had has been re-introduced, where quickshell crashes regardless of if I input a blank password or correct/incorrect one, only fingerprint is working.

I'll revert back to upstream to check whether this PR did that for me, or if something else changed for that issue to have come back

[bonnjalal]

I think I have a bug in the PR that may cause this problem and It only happens for users (like you) who have both fprint and Howdy available. onUnlock function was trying to abort fprint even when it wasn't running (because Howdy was used), which may cause the crash.

function onUnlock(): void {
        fprint.abort();
        howdy.abort();
    }

should be:

function onUnlock(): void {
    if (fprint.active)
        fprint.abort();

    if (howdy.active)
        howdy.abort();
}

Even if this is not the problem, I think it should be fixed, so I will add a new commit right now.

[PixelKhaos]

That's not all, it's a bit more complicated than that it seems. This is an issue for me upstream too.

So, I have been able to login with password just fine in cases where fprint wasn't active, due to hibernation or other fprint error causing it to not be active. But in testing this I manually lock, where fprint then is active immediately.
If I then try to auth with anything but fprint, quickshell crashes.

It used to do it even with fprint erroring out and not active, but that was resolved at some point, I just never noticed that trying to use password when fprint is active or failed retries doesn't work

Edit: So, it's 100% that fprint PAM context does not work with password simultaneously, disabling fprint just works. So need to handle dropping fprint somehow when trying to auth with password, which most of my attempts didn't work lmao

[bonnjalal]

That's not all, it's a bit more complicated than that it seems. This is an issue for me upstream too.

So, I have been able to login with password just fine in cases where fprint wasn't active, due to hibernation or other fprint error causing it to not be active. But in testing this I manually lock, where fprint then is active immediately. If I then try to auth with anything but fprint, quickshell crashes.

It used to do it even with fprint erroring out and not active, but that was resolved at some point, I just never noticed that trying to use password when fprint is active or failed retries doesn't work

Edit: So, it's 100% that fprint PAM context does not work with password simultaneously, disabling fprint just works. So need to handle dropping fprint somehow when trying to auth with password, which most of my attempts didn't work lmao

Hi @PixelKhaos, I think You're right, the crash is because fprint auto-starts, and then the code tries to run passwd or howdy at the same time.

I just pushed a new commit that fixes this.

• It stops fprint (and howdy) from auto-starting, which implements the "press Enter for biometrics" feature.

• The handleKey function now explicitly aborts any running biometric scan (like fprint) before it starts a new one (like passwd or howdy).

This ensures only one PAM service runs at a time. Could you pull the new commit and test it? I think this should fix the crash.

The new function:

// *** FIX: We no longer block if fprint/howdy are active, because we need to interrupt them later. ***
        if (passwd.active)
            return;
        if (event.key === Qt.Key_Enter || event.key === Qt.Key_Return) {
            if (buffer.length > 0) {
                // *** FIX: Abort all biometrics first ***
                if (fprint.active)
                    fprint.abort();
                if (howdy.active)
                    howdy.abort();

                passwd.start();
            } else {
                if (howdy.available) {
                    // *** FIX: Abort fprint before starting howdy ***
                    if (fprint.active)
                        fprint.abort();
                    howdy.start();
                } else if (fprint.available) {
                    // *** FIX: Abort howdy before starting fprint ***
                    if (howdy.active)
                        howdy.abort();
                    fprint.start();
                }
            }
        }

[PixelKhaos]

Yeah, did some further tests, and realized that removing all instances of fprint.abort() actually "fixes" the issue.
It stays active after password auth until it times out on its own, but at least no crash.
So the issue lies entirely within how fprint.abort() is crashing quickshell, regardless of how and when it's called it seems, while a delayed call of it after sucessful login does let you in without being locked out, it does eventually crash inside hyprland.

I'll try your new update, but I wouldn't worry too much about this, as it has not really anything to do with your Howdy implementation, sorry for the hijack here haha. Howdy worked great without fprint!

[soramanew]

Thanks @bonnjalal for the nice PR! I like the addition of Howdy integration, however having the PAM contexts be exclusive to each other would kinda defeat the purpose of having multiple of them in the first place. Also, we probably don't need to have a trigger for the Howdy context, it should just run when the lock is triggered like the fprint context.

That crash is interesting, from what I can understand of your and @PixelKhaos messages, it seems that calling fprint.abort() is crashing Quickshell? In that case, does only aborting it if it's active help? Also, running multiple PAM contexts simultaneously shouldn't affect each other, so it shouldn't be the passwd PAM context conflicting with the fprint PAM context causing the crash.

[PixelKhaos]

That crash is interesting, from what I can understand of your and @PixelKhaos messages, it seems that calling fprint.abort() is crashing Quickshell? In that case, does only aborting it if it's active help? Also, running multiple PAM contexts simultaneously shouldn't affect each other, so it shouldn't be the passwd PAM context conflicting with the fprint PAM context causing the crash.

@soramanew I've tried with some different lenghts of delaying to call abort, and can confirm that regardless of if the reader is in an active input state or not, it will crash quickshell.

• After 15+ seconds (well past fprintd timeout), fprint.active is still true
• PAM reports "Authenticated successfully" (session completed)
• Calling abort() on this stuck state crashes with SIGABRT and AMD GPU errors
• Other PAM modules (Howdy, password) don't have this issue

From my .qslog I could see that the auth completed succesfully, normally like

quickshell.service.pam
Got message from subprocess.
...
Subprocess exited with code 0

Then 15 seconds later, I see

Testing delayed abort - fprint.active: true
Calling fprint.abort() on active session
Killing subprocess for PamConversation(0x7ff63f0b92a0)

In the case of running it as usual without any delay, I think it still tries to abort a dead process?
Unless its in that case due to some other transitional, but also caused by calling abort regardless

[soramanew]

Hmm, this might be a quickshell or fprint bug. I'll see what I can dig up from the quickshell source

[PixelKhaos]

Hmm, this might be a quickshell or fprint bug. I'll see what I can dig up from the quickshell source

Yeah, I assumed as such since it seems to work absolutely fine with any other PAM contexts, and also works totally fine in waking up in cases where fprint service didnt start/was available, as then it just never would call abort on it so missed it for a long time.

But I would say, not using abort for fprint is actually fine from what I can tell, the only actual drawback is cosmetic, as my reader lights up green for a while after unlocking, but does not affect usability and it times out on its own eventually too. So the need for aborting is pretty minor

edit: oh and here's what I got out of qslog from a crash without delay added

Got message from subprocess.
4Sending response for PamConversation(0x7f26144e16c0)
Subprocess exited with code 0
Destroyed WlDmaBuffer...
Destroyed GBM device...
WlDmaBufferQSGTexture destroyed.
6Killing subprocess for PamConversation(0x7f261a34b120)

Not really offering much more context other than that it destroyed some graphics then killing the pam subprocess.
so either already exited but active is stuck at true, or is in a transitional state during unlock

[bonnjalal]

Hi @soramanew and @PixelKhaos,

Thanks for the great discussion. I've been testing this on a separate branch to find the best way forward.

First, I'll revert the handleKey changes I made. It's clear the crash @PixelKhaos is seeing when fprint is active is a separate concurrency issue.

This brings us to the core design choice for Howdy. The main challenge is that fprint and howdy are technically very different:

• fprint is passive: It auto-starts and just idles forever, waiting for a touch.
• howdy is active: It turns on the camera to scan, but it's designed to time out.

Based on my testing, I see two possible paths we can take, and I'd like your opinion on which one I should implement.

Path 1: Implement Full Auto-Start

I can follow your request for auto-start by forcing Howdy into a continuous loop (making the QML re-call howdy.start() every time it fails or times out).

• Pro: This works and feels like Windows Hello. The user just looks at the screen and it unlocks.
• Con: My tests show this is not practical. The loop makes the camera scan constantly (about 40 times or more just until the screen goes black). It keeps scanning even when the screen is black (DPMS). I think this will cause a massive battery drain on laptops.

Path 2: Implement as a Manual "On-Demand" Service

We can treat Howdy as a manual-trigger service because it's so active.

• Pro: This completely solves the battery drain and DPMS problem.
• Con: It's not the fully automatic experience that fprint provides.

Implementation: Howdy would not auto-start. Instead, I would add a UI element (like the clickable "eye" icon @PixelKhaos suggested, and also can keep the Enter shortcut) that the user must press to begin the face scan.

Given these findings, what are your thoughts? I'm happy to go with Path 1, or I can build Path 2.

Let me know which road you think is best for the project.

[PixelKhaos]

I guess another issue with passive Howdy is if you manually lock, you may accidentally unlock it right away if you're in front of the camera still, but then on the other end the reason for Howdy is to be very convenient of just looking at the screen to unlock.

You could add mouse movement detection to trigger Howdy similar to how movement wakes the monitor.

In LockSurface.qml, add a MouseArea with hoverEnabled that tracks position changes. Use a Timer to delay enabling hover detection by ~100ms to avoid triggering on lock screen appearance. In onPositionChanged, check if movement exceeds a threshold etc

In Pam.qml, add a function that checks if Howdy is available and not already running, then calls howdy.start(). Add a howdyTriggeredByActivity flag to only trigger once per lock session (reset in onSecureChanged), that way it'd be semi-passive. The only question then is how you handle the usual retries, etc.

But, maybe just overthinking it, allowing it to simply run automatically but needing user interaction of some kind to simply reset the retries rather than retrying forever? Though, I do normally set my dpms off to like 600 seconds, so that duration of the camera checking (at very low resolution and framerate, for only 4 seconds) might not be that much of a battery impact to really worry?

[bonnjalal]

Hi @PixelKhaos, thanks for that super detailed breakdown.

The accidental unlock is a small risk (I think we can make a delay between when the screen locked and the howdy starts), but the convenience of just looking at the screen is the whole point. I think your last suggestion is the perfect path forward.

Here is my new plan based on your feedback:

• I'll implement the auto-start for Howdy with the continuous loop. This will give the convenient, "always scanning" feel we're looking for.
• I'll hook into the lock screen's state. I'll make the Howdy loop abort as soon as the screen idles/dims (when DPMS kicks in) and restart it when the screen wakes up (e.g., on mouse move).

[soramanew]

You can look in IdleMonitors.qml to hook on the idle timeouts, as that is where it's handled

[bonnjalal]

@soramanew @PixelKhaos
Please check the new commits I pushed, I think know it work as expected.

[PixelKhaos]

I'll look more into any errors etc when I can, but with this latest update quickshell crashes on fingerprint or password auth while Howdy is active, tried removing the fprint abort calls, and it still crashes so its something else there.

[bonnjalal]

I'll look more into any errors etc when I can, but with this latest update quickshell crashes on fingerprint or password auth while Howdy is active, tried removing the fprint abort calls, and it still crashes so its something else there.

That's weird, for me I can login with password just fine even when howdy is running.

Try to abort the fprint and howdy before pressing enter for password and see if this can help fix you "password" problem.

if (event.key === Qt.Key_Enter || event.key === Qt.Key_Return) {
    if (fprint.active)
        fprint.abort();
    if (howdy.active)
        howdy.abort();

    passwd.start();
}

I'm using NixOs by the way