References and tools
The short list of what AdviceIT directly builds on: the dataset the advisors are trained on, the handful of works behind the design of the study and the explanations, and the software and models it runs on. Where a work informed a specific part of the app, that part is named.
Dataset
- Bonelli, M. (2026). ILS-Bench: Investor Language-to-Suitability Benchmark. Mendeley Data, V1. https://doi.org/10.17632/w48mh2dtg5.1. Licence CC BY 4.0. Training data for the neural network and the interpretable model, the shared label vocabulary, the Human review outcome, the case browser and the language-reading benchmark. See the Training data page.
Robo-advisory, risk and ethics
- Nahidi, N., and Zarifis, A. (Eds.) (2026). AI, FinTech, and the Future of Robo-Advisory: Risk Management and Ethical Considerations. Contributions to Finance and Accounting. Springer Nature Switzerland. https://doi.org/10.1007/978-3-032-18109-1. Domain background on AI-driven robo-advisory, its risk management and its ethical questions: the framing of suitability, of escalation to a human adviser, and of the ethics note and debrief in the study design.
Provenance
- Pratama, R., and co-authors (2026). Systematic literature review of trust and algorithm aversion in the choice between human and AI financial advisors. Presented at SSRAAI 2026. The review that identified miscalibrated trust as the problem. AdviceIT is its design-side follow-up.
Trust, reliance and algorithm aversion
- Lee, J. D., and See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50 to 80. The notion of appropriate reliance and calibrated trust that the dependent variables operationalise.
- Dietvorst, B. J., Simmons, J. P., and Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114 to 126. Under-reliance, and the reason the flawed scenario is debriefed.
- Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., and Weld, D. S. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. CHI 2021. Explanations can increase reliance on wrong advice, the reason sound and flawed trials are both needed.
Explainable AI methods
- Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1, 206 to 215. The interpretable-by-design position behind the interpretable rule-based advisor and the fidelity factor.
- Lundberg, S. M., and Lee, S.-I. (2017). A unified approach to interpreting model predictions. NeurIPS 2017. SHAP: Shapley values as feature attributions relative to a baseline, computed exactly for the neural network.
- Wachter, S., Mittelstadt, B., and Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law and Technology, 31(2). The counterfactual content: smallest change that alters the outcome, found by search, and the contrastive "why not X" panel.
- Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017). On calibration of modern neural networks. ICML 2017. Temperature scaling and expected calibration error for the probabilities shown in the confidence content.
Human-centred explanation interfaces and measures
- De Croon, R., and colleagues, Augment research group, KU Leuven (2025). Designing and personalising hybrid health explanations for lay users. ACM Transactions on Interactive Intelligent Systems. The hybrid content and the adaptive delivery, and personalisation to a measured user characteristic.
- Lusardi, A., and Mitchell, O. S. (2011). Financial literacy around the world: An overview. Journal of Pension Economics and Finance, 10(4), 497 to 508. The "Big Three" financial literacy questions used as the moderator and by the adaptive delivery.
Software, models and services
| What | Used for | Licence and source |
|---|---|---|
| Python 3 (standard library) | serve.py, the study server, and ml/fetch_ils_bench.py, tools/make_pages.py | PSF licence, python.org |
| NumPy | Training both advisors in ml/train_model.py (multilayer perceptron with Adam, multinomial logistic regression, temperature scaling) | BSD, numpy.org |
| openpyxl | Reading the ILS-Bench Excel file in ml/fetch_ils_bench.py | MIT, openpyxl.readthedocs.io |
| WebLLM (MLC project) | Running the language model in the browser for the conversational condition, the narrative reading and the benchmark. Ruan, C. F., and colleagues (2024). WebLLM: A high-performance in-browser LLM inference engine. arXiv:2412.15803. | Apache 2.0, github.com/mlc-ai/web-llm, loaded from jsDelivr (esm.run) |
| Qwen 2.5 1.5B Instruct | Default language model for the conversational delivery and the narrative reading | Apache 2.0, Alibaba Cloud, weights via Hugging Face (MLC builds) |
| Llama 3.2 1B Instruct | Lighter alternative language model | Llama 3.2 Community License, Meta Platforms, weights via Hugging Face (MLC builds) |
| WebGPU | GPU access in the browser for WebLLM | W3C specification, w3.org/TR/webgpu |
| Mendeley Data | Hosting of the ILS-Bench dataset and its API used by the fetch script | data.mendeley.com |
| GitHub and GitHub Pages | Source hosting and static deployment | pages.github.com |
| Node.js and jsdom | Development only: automated checks of the pages, the explanations and the study flow | MIT, github.com/jsdom/jsdom |
| Browser standards | localStorage for the session log, Blob and object URLs for downloads, the dialog element for the panels, ARIA for accessibility | WHATWG and W3C |
| WCAG 2.1 level AA | Contrast, keyboard operation, focus visibility, labels, never colour alone | w3.org/TR/WCAG21 |
Everything else in AdviceIT (the label rules, the explanation modules, the study machinery, the pages) is original code written for this project, released under the MIT licence.