The use and application of Artificial Intelligence (AI) for business process improvement and management is often discussed, especially due to the potentially negative impact if AI is implemented in the wrong way. We discuss the use of AI for the management and improvement of business processes in the financial domain, and how to ensure responsible AI use in enterprises for this aim, and how can one create associated audit to enforce principles of fariness in such AI implementations. We reflect on wider societal challenges, potential solutions for financial regulation and policy-makers, and we outline considerations for AI and data protection regulation for companies, while also considering the technical challenges that would need to be solved.