Enterprise Architecture Framework for Secure Large Language Model Deployment in Financial Services
DOI:
https://doi.org/10.63282/3117-5481/AIJCST-V6I1P109Keywords:
Large Language Models, Enterprise Architecture, Financial Services, Retrieval-Augmented Generation, AI Governance, Responsible AIAbstract
The rapid advancement of Large Language Models (LLMs) has created new opportunities for financial institutions to enhance customer engagement, investment research, advisor productivity, and operational efficiency. However, enterprise adoption of LLMs remains challenging due to stringent regulatory requirements, data privacy concerns, cybersecurity risks, model governance complexities, and integration with existing enterprise systems. While existing research primarily focuses on LLM capabilities and model performance, limited attention has been given to a comprehensive enterprise architecture that enables secure, scalable, and governed deployment of LLMs within regulated financial environments. This paper proposes a novel Enterprise Architecture Framework for Secure Large Language Model Deployment in Financial Services to support the responsible adoption of generative AI across banking, asset and wealth management, and capital markets. The proposed framework consists of six vertically integrated architectural layers—Experience and Channels, AI Orchestration and Policy, LLM Services, Knowledge and Retrieval-Augmented Generation (RAG), Enterprise Integration, and Platform and Deployment—supported by four cross-cutting control domains: Security, Governance, Compliance, and Observability. These layers provide a scalable foundation for integrating LLM capabilities into enterprise applications while ensuring regulatory compliance, data protection, explainability, and operational resilience. A design science research methodology is employed to develop and validate the framework using a representative enterprise financial services case study. The evaluation demonstrates how the proposed architecture enables secure knowledge retrieval, intelligent research assistance, workflow automation, and decision support while maintaining identity-based access control, human oversight, auditability, and responsible AI governance. By combining cloud-native architecture, API-driven integration, zero-trust security, and continuous monitoring, the framework addresses critical enterprise requirements for secure LLM adoption. The proposed framework contributes a reusable enterprise reference architecture that bridges the gap between generative AI innovation and the operational, security, and governance demands of modern financial institutions, providing enterprise architects and technology leaders with practical guidance for implementing trustworthy and scalable LLM solutions.
References
[1] Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., et al.: AgentBench: Evaluating LLMs as agents. arXiv:2308.03688 (2023).
[2] Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., Chen, E.: A survey on multimodal large language models. arXiv:2306.13549 (2023).
[3] Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Proceedings of the International Conference on Machine Learning, 19730-19742 (2023).
[4] Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H.B., Patel, S., Ramage, D., Segal, A., Seth, K.: Practical secure aggregation for privacy-preserving machine learning. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, 1175-1191 (2017).
[5] Guha, N., Nyarko, J., Ho, D.E., Ré, C., Chilton, A., Narayana, A., et al.: LegalBench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural Information Processing Systems 36 (2023).
[6] Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., et al.: Holistic evaluation of language models. Transactions on Machine Learning Research (2023).
[7] Islam, P., Kannappan, A., Kiela, D., Qian, R., Scherrer, N., Vidgen, B.: FinanceBench: A new benchmark for financial question answering. arXiv:2311.11944 (2023).
[8] Tabassi, E.: Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology (2023). https://doi.org/10.6028/NIST.AI.100-1
[9] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in Neural Information Processing Systems 30 (2017).
[10] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al.: Language models are few-shot learners. Advances in Neural Information Processing Systems 33, 1877-1901 (2020).
[11] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., et al.: On the opportunities and risks of foundation models. arXiv:2108.07258 (2021).
[12] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33, 9459-9474 (2020).
[13] Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.-t.: Dense passage retrieval for open-domain question answering. Proceedings of EMNLP, 6769-6781 (2020).
[14] Izacard, G., Grave, E.: Leveraging passage retrieval with generative models for open-domain question answering. Proceedings of EACL, 874-880 (2021).
[15] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., et al.: Retrieval-augmented generation for large language models: A survey. arXiv:2312.10997 (2023).
[16] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations (2023).
[17] Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., et al.: Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36 (2023).
[18] Araci, D.: FinBERT: Financial sentiment analysis with pre-trained language models. arXiv:1908.10063 (2019).
[19] Wu, S., Irsoy, O., Lu, S., Dabravolski, V., Dredze, M., Gehrmann, S., et al.: BloombergGPT: A large language model for finance. arXiv:2303.17564 (2023).
[20] Yang, H., Liu, X.-Y., Wang, C.D.: FinGPT: Open-source financial large language models. arXiv:2306.06031 (2023).
[21] Rose, S., Borchert, O., Mitchell, S., Connelly, S.: Zero Trust Architecture. NIST Special Publication 800-207. National Institute of Standards and Technology (2020). https://doi.org/10.6028/NIST.SP.800-207
[22] Souppaya, M., Scarfone, K., Dodson, D.: Secure Software Development Framework (SSDF), Version 1.1. NIST Special Publication 800-218. National Institute of Standards and Technology (2022). https://doi.org/10.6028/NIST.SP.800-218
[23] OWASP Foundation: OWASP Top 10 for Large Language Model Applications, Version 1.0 (2023).
[24] International Organization for Standardization: ISO/IEC 27001:2022 Information security, cybersecurity and privacy protection - Information security management systems - Requirements. ISO, Geneva (2022).
[25] International Organization for Standardization: ISO/IEC 42001:2023 Information technology - Artificial intelligence - Management system. ISO, Geneva (2023).
[26] European Parliament and Council: Regulation (EU) 2016/679, General Data Protection Regulation. Official Journal of the European Union (2016).
[27] United States Congress: Gramm-Leach-Bliley Act, Pub. L. 106-102, 113 Stat. 1338 (1999).
[28] PCI Security Standards Council: Payment Card Industry Data Security Standard, Version 4.0 (2022).
[29] Financial Stability Board: Artificial intelligence and machine learning in financial services: Market developments and financial stability implications. FSB (2017).
[30] Basel Committee on Banking Supervision: Principles for operational resilience. Bank for International Settlements (2021).
[31] Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Quarterly 28(1), 75-105 (2004).
[32] Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research methodology for information systems research. Journal of Management Information Systems 24(3), 45-77 (2007).
[33] Gregor, S., Hevner, A.R.: Positioning and presenting design science research for maximum impact. MIS Quarterly 37(2), 337-355 (2013).
[34] International Organization for Standardization: ISO/IEC 25010:2011 Systems and software engineering - Systems and software quality requirements and evaluation - System and software quality models. ISO, Geneva (2011).
[35] Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., et al.: Ethical and social risks of harm from language models. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 214-229 (2022).
[36] Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S.: On the dangers of stochastic parrots: Can language models be too big? Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 610-623 (2021).
[37] Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., et al.: Model cards for model reporting. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 220-229 (2019).
[38] Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Daumé III, H., Crawford, K.: Datasheets for datasets. Communications of the ACM 64(12), 86-92 (2021).
[39] Ribeiro, M.T., Singh, S., Guestrin, C.: 'Why should I trust you?': Explaining the predictions of any classifier. Proceedings of the ACM SIGKDD Conference, 1135-1144 (2016).
[40] Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30 (2017).
[41] Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., et al.: Extracting training data from large language models. 30th USENIX Security Symposium, 2633-2650 (2021).
[42] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M.: Not what you've signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. Proceedings of the ACM Workshop on Artificial Intelligence and Security, 79-90 (2023).
[43] Zou, A., Wang, Z., Kolter, J.Z., Fredrikson, M.: Universal and transferable adversarial attacks on aligned language models. arXiv:2307.15043 (2023).
[44] Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., et al.: Red teaming language models with language models. Proceedings of EMNLP, 3419-3448 (2022).
[45] Manakul, P., Liusie, A., Gales, M.J.F.: SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. Proceedings of EMNLP, 9004-9017 (2023).
[46] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-t., Koh, P.W., Iyyer, M., Zettlemoyer, L., Hajishirzi, H.: FActScore: Fine-grained atomic evaluation of factual precision in long-form text generation. Proceedings of EMNLP, 12076-12100 (2023).
[47] Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., et al.: Survey of hallucination in natural language generation. ACM Computing Surveys 55(12), 1-38 (2023).
[48] Mialon, G., Dessì, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., et al.: Augmented language models: A survey. Transactions on Machine Learning Research (2023).
[49] Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations (2022).
[50] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., et al.: Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35, 27730-27744 (2022).
[51] Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., et al.: Constitutional AI: Harmlessness from AI feedback. arXiv:2212.08073 (2022).
[52] OpenAI: GPT-4 technical report. arXiv:2303.08774 (2023).
[53] Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288 (2023).
[54] McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. Proceedings of AISTATS, 1273-1282 (2017).
[55] Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A.N., et al.: Advances and open problems in federated learning. Foundations and Trends in Machine Learning 14(1-2), 1-210 (2021).
[56] European Commission High-Level Expert Group on Artificial Intelligence: Ethics guidelines for trustworthy AI. European Commission (2019).
[57] Organisation for Economic Co-operation and Development: Recommendation of the Council on Artificial Intelligence. OECD/LEGAL/0449 (2019).
[58] National Institute of Standards and Technology: NIST Privacy Framework: A tool for improving privacy through enterprise risk management, Version 1.0 (2020).
[59] National Institute of Standards and Technology: Cybersecurity Framework, Version 1.1. NIST (2018).
[60] Financial Stability Board: Enhancing third-party risk management and oversight: A toolkit for financial institutions and financial authorities. FSB (2023).
