Kolmogorov-Arnold Networks for Alzheimer's Disease Detection using Multimodal Speech Analysis

Introduction

Changes in speech and language can emerge before the clinical diagnosis of Alzheimer's disease, making spoken-language analysis a promising non-invasive and scalable screening approach.

Speech-based AD detection remains difficult for three reasons: labeled datasets are small, complex end-to-end models can overfit and require substantial computation, and clinical-facing systems benefit from interpretable and auditable decision mechanisms.

Motivation

Freezing large pretrained encoders can reduce the number of trainable parameters and improve stability in low-data settings. The classifier head still has to learn effective nonlinear decision boundaries from fixed audio and text representations.

Kolmogorov-Arnold Networks use learnable spline-based functions instead of fixed activation functions. Prior speech work suggests that placing a KAN block between two feedforward layers may provide a particularly effective configuration.

Objective

Primary objective: determine whether a Feedforward-KAN-Feedforward classifier head can provide more accurate and stable Alzheimer's disease classification than conventional MLP and pure KAN heads under comparable low-parameter budgets.

Specific objectives:

1. Compare MLP, KAN, and FKF heads across audio-only, transcript-only, and multimodal settings.
2. Evaluate Small, Medium, and Large classifier-head capacity tiers using aligned subject-independent cross-validation.
3. Investigate whether FKF performance depends on the ordering of its feedforward and KAN components.
4. Examine training dynamics, internal activation flow, and learned spline functions to explain FKF behaviour.

Methods

Dataset and Evaluation

The evaluation used ADReSSo 2021: 237 participants, including 122 with probable AD and 115 cognitively normal controls.

Binary AD-versus-control classification was evaluated using aligned, stratified, subject-independent five-fold cross-validation. Macro-F1 and ROC-AUC were reported, with participant-level splitting used to prevent data leakage.

Audio and Transcript Processing

Audio recordings were cleaned, converted to mono, resampled to 16 kHz, and divided into overlapping 10-second windows with 50% overlap. Low-energy windows were removed.

Transcripts were generated automatically using Whisper-large-v3 and divided into fixed-length 64-token chunks.

Proposed Architecture

Frozen wav2vec2 and DistilBERT encoders produced audio-window and transcript-chunk embeddings. Lightweight classifier heads generated local predictions, which were averaged to obtain participant-level audio and text outputs. A learned late-fusion module combined both modalities.

Only the classifier and fusion heads were trained; the pretrained encoders remained frozen.

Classifier Head Comparison

Three classifier families were evaluated at Small, Medium, and Large parameter tiers:

• MLP: conventional feedforward classifier
• KAN: pure spline-based Kolmogorov-Arnold Network
• FKF: Feedforward-KAN-Feedforward

A small-tier audio ablation also compared MLP, KAN, FKF, Feedforward-KAN, and KAN-Feedforward.

Results

Best Late-Fusion Result

Overall Performance

FKF was consistently strong across audio-only, transcript-only, and multimodal settings. Learned late fusion improved over the unimodal outputs and was the best-performing fusion strategy in eight of nine evaluated settings.

Performance Under Limited Capacity

The clearest FKF advantages appeared under tight classifier-head budgets.

Small-tier fusion macro-F1:

• FKF: 0.8068 ± 0.0235
• KAN: 0.7812 ± 0.0556
• MLP: 0.7015 ± 0.1067

Small FKF models were frequently competitive with larger alternatives, supporting a structural advantage rather than an effect of parameter count alone.

Ordering Ablation

FKF achieved both the highest macro-F1 and the lowest fold-to-fold variability, showing that the ordering of the feedforward and KAN components matters.

Model definitions:

• MLP: conventional multilayer perceptron
• KAN: pure Kolmogorov-Arnold Network
• FKF: Feedforward-KAN-Feedforward
• FK: Feedforward-KAN
• KF: KAN-Feedforward

Discussion

The FKF ordering appears to matter because FF1 conditions frozen features, KAN learns spline nonlinearities, and FF2 produces stable logits. The first feedforward layer mixes and rescales the frozen encoder representations before nonlinear modeling, and the final feedforward layer recombines the KAN outputs into stable class predictions.

Mechanistically, FKF showed smoother convergence and lower across-fold variability than alternative orderings. Activation analysis indicated progressive nonlinear engagement and class-margin formation, while representative spline functions remained low-complexity and interpretable.

Conclusion

• Frozen audio and text encoders paired with lightweight FKF heads provide strong low-parameter AD classification.
• FKF was the most consistent head family, with its clearest advantages appearing under tight parameter budgets.
• Learned late fusion combined complementary acoustic and linguistic information, reaching 0.8556 macro-F1 and 0.9220 ROC-AUC.

Future Work

External validation, lower-cost ASR, temporal and token-level interpretability, and richer multimodal fusion.

References

1. Luz S, Haider F, de la Fuente S, Fromm D, MacWhinney B. Detecting Cognitive Decline Using Speech Only: The ADReSSo Challenge. Interspeech, 2021.
2. Liu Z, Wang Y, Vaidya S, et al. KAN: Kolmogorov-Arnold Networks. 2024.
3. Koudounas A, La Quatra M, Pastor E, Siniscalchi SM, Baralis E. "KAN You Hear Me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding. Interspeech, 2025.
4. Baevski A, Zhou H, Mohamed A, Auli M. wav2vec 2.0. NeurIPS, 2020.
5. Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT. 2019.

Acknowledgements

This work was conducted at Wilfrid Laurier University. The authors gratefully acknowledge institutional support for this research. Computational resources were provided by the Digital Research Alliance of Canada.