Leveraging Generative Artificial Intelligence for Financial Market Trading Data Management and Prediction

1. Introduction

In recent years, artificial intelligence technology has been rapidly iterated and deeply integrated with the financial industry and has been widely used in multiple scenarios, such as intelligent customer service, credit scoring, intelligent investment advisory, and risk management. Financial authorities such as central banks are also exploring the use of AI to enhance economic monitoring, analysis, and forecasting. Artificial intelligence (AI) technology is evolving from “analytical” to “generative” [1]. Analytical AI (Analytical AI) uses machine learning algorithms to calculate the conditional probability distribution in the data according to the existing data set for the model’s analysis, judgment, and prediction, mainly used in advertising recommendation, auxiliary decision, data analysis, and other fields. However, analytical AI can not “out of nothing” and can only handle specific tasks based on particular rules, but can not create anything new, so it is also called “narrow AI” or “weak AI.”

Generative AI trains on a more comprehensive data set, learning the underlying logic and generating new data sets to create new content such as articles, pictures, music, computer code, and more. Compared with analytical AI, generative AI has a more robust understanding, reasoning, and generation capabilities and a more comprehensive range of application scenarios [2,3,4,5]. The most representative generative AI is the large language model; OpenAI’s ChatGPT, Baidu’s Wenxin Yiyi, Ali’s Tongyi Qianwen, etc., all belong to this category. More than 200 large models are worldwide, and Chinese and American technology companies have developed 90%.

Data management and forecasting are central to ensuring trading security and optimizing investment decisions in financial markets. Through its powerful data generation and analysis capabilities, generative artificial intelligence technology can significantly improve the management efficiency of economic data and provide more accurate market forecasts. This not only helps to increase investment returns but also effectively reduces risk. In this context, it becomes essential to explore the application of generative AI in financial data management and forecasting.

2. Related Work

2.1. Financial Market Data Management

AI is a natural fit for the financial industry. Data is the essential input of AI, both analytical and generative AI, and can not be separated from a large number of data training [6]. The financial industry has a high degree of digitalization. It has accumulated a lot of user and transaction data in daily business, making it one of the ideal scenarios for AI applications. Financial institutions also generally use AI technology to upgrade or innovate financial business. According to the Business Insider survey, 80% of banks in the United States believe that AI can help improve financial services and have or plan to integrate AI with financial business [7,8,9]. By 2027, the global AI financial services market is expected to grow to $130 billion.

Financial institutions mainly use analytical AI to assist human employees. The application scenarios are as follows:

1. Intelligent customer service. The work of customer service has prominent procedural characteristics, so it is one of the most widely used scenarios for AI Intelligent customer service mainly undertakes two tasks [10]: First, answer customers’ questions and doubts. For example, Erica, the smart customer service of Bank of America, uses voice recognition and natural language processing technology to analyze customers’ problems, extract critical information, and provide corresponding solutions; Cimb’s Eva specifically addresses the needs of SMEs and can work 7*24 hours a day, playing a more significant role in helping SMEs during the epidemic. Ping An Bank of China’s AI customer service has undertaken 80% of the workload. Second, personalized business recommendations [11]. For example, Wells Fargo cooperated with Google to analyze user characteristics based on transaction data, social data, etc., and recommend financial services or investment products in a targeted manner.

2. Credit score. Traditional credit scoring models primarily use structured data, which makes it impossible to score a customer if they do not have a bank account and the associated transaction transfers [12,13,14,15,16,17]. AI can widely use structured (such as transaction history) and unstructured data (such as employment history, spending habits, etc.) for credit scoring, significantly improving efficiency, accuracy, and reach. For example, the United Bank of the Philippines has built a credit scoring model based on artificial intelligence technology to give credit ratings to unbanked groups, thereby improving loan availability.

3. Smart Advisors [18]. An intelligent advisory system built with AI technology can generate a portfolio suitable for investors’ needs by analyzing investors’ risk appetite, financial goals, and market conditions and adjusting investment strategies promptly according to market operation [19]. This helps to lower the investment threshold and provide more precise investment choices for investors with different levels of wealth and risk tolerance. Some Chinese and foreign financial institutions, such as JPMorgan Chase in the US and Flush in China, have launched intelligent advisory services.

4. Risk management. Risk management models based on AI technology can analyze, identify, and predict risk factors in finance, investment, credit, and other fields through extensive data analysis and take measures to reduce risks and protect the interests of financial institutions and consumers [20,21,22,23,24]. For example, Singapore’s DBS bank has used AI to improve its anti-money laundering/anti-terrorist financing alarm prioritization, significantly reducing the number of false positives. Some commercial banks in China have established AI anti-money laundering models, using machine learning, knowledge graphs, and other technologies for real-time monitoring and intelligent analysis of anti-money laundering.

2.2. Generative Artificial Intelligence (GAN) Technology

As a new type of production mode, generative artificial intelligence (AI) has the technical and economic characteristics of complementarity, intelligence, integration, and creativity [25]. Artificial intelligence has penetrated financial products, business models, business processes, and other links, forming enabling effects, economies of scale, economies of scope, and flywheel effects, resulting in cost reduction, efficiency increase, value creation, and business format innovation. The deep integration of generative artificial intelligence and the financial industry is based on technology, with data as the core, computing power as the support, algorithm as the drive, and rules as the guarantee [26]. It has some auxiliary value for financial business. Still, it is only possible to partially subvert the financial industry’s traditional paradigm. The application of generative artificial intelligence in the financial field faces many problems and challenges, and the interoperability between artificial intelligence technology and the financial sector should be strengthened.

1. Generate Adversarial Networks (Gans)

Generative Adversarial Networks (Gans) are one of the most popular research fields in machine learning in recent years. GAN was proposed by Ian Goodfellow and others in 2014. It uses a new training method to enable the generated model to learn the data distribution and create samples that are indistinguishable from the actual data. The core idea of GAN is to play games with two competing network models - Generator and Discriminator, and finally achieve the goal of generating realistic samples by generator [27].

GANs consist of two independent adversarial networks: generator and discriminator. When only a noisy image array is used as input, the generator is trained to create realistic images. The discriminator is trained to classify whether an image is actual or not.

The true power of GANs comes from the adversarial training model they follow. The generator’s weight is learned based on the loss of the discriminator. As a result, the generator is trained by the images it generates, and it is difficult to know whether the generated images are real or fake. At the same time, the resulting pictures look more and more realistic, and the discriminator becomes more and more able to tell whether an image is actual or not, no matter how similar the image looks to the naked eye.

From the technical point of view, the discriminator’s loss is the error value of classifying the image as true or false. We’re measuring its ability to distinguish between real and fake photos. The generator’s loss will depend on its ability to “fool” the discriminator with the phony image, i.e., the discriminator only misclassifies the fake image because the generator wants the value to be as high as possible.

So, GANs build a feedback loop where the generator helps train the discriminator, and the discriminator helps train the generator. They get stronger at the same time. The chart below helps illustrate this point.

Figure 1. Generate an adversarial network architecture diagram.

Figure 1. Generate an adversarial network architecture diagram.

2. Variational Autoencoder (VAE)

Variations, or variations. What should we know about functionals before we talk about variations? Reviewing the functions we have learned since the beginning involves taking a given input value, x, through a series of changes, f(x), to get the output value y. Notice here that we’re putting a number in, and we’re putting a number out. Is there a case where our argument is a function instead of a number? [28]. The classic question is, given two fixed points, A and B, we can take any path from point A to point B and find out in what path the shortest time from point A to point B is. Most people have the answer by this point - the shortest line between two points. A function where the input variable is a function and the output variable is a numerical value is called a function. The popular understanding of a function is a function of a function.

Usually, we feed the input image into the NN Encoder and get a latent code. Usually, the dimension of this latent code is much smaller than the dimension of the input object, which is a compact representation of the input object. Next, we feed this latent code into [8] NN Decoder for decoding and outputting the reconstructed original object.

Figure 2. Auto-Encoder architecture diagram.

Figure 2. Auto-Encoder architecture diagram.

Auto-Encoder was proposed by Rumelhart in 1986 and can be used for processing high-dimensional complex data, promoting neural network development. A self-coding neural network is an unsupervised learning algorithm (training examples are not labeled) that uses the BP backpropagation algorithm and strives to make the output as close to the input as possible.

AE networks generally have two characteristics [29]:

1. dim(Hidden layer) << dim(Input layer): the hidden layer dimension should be much smaller than the input dimension.

2. The Output of the decoding layer is used for Reconstruction Input, so we should minimize (Reconstruction error(Input, Output)), that is, minimize the reconstruction error between input and output.

VAE can be applied to data dimensionality reduction, feature extraction, and data visualization analysis in machine learning, and it can also be extended and applied to generative models.

As generative artificial intelligence technologies, generative adversarial networks (Gans) and variational autoencoders (VAE) are essential in financial market risk monitoring and management. By generating adversarial methods, Gans can synthesize high-quality financial data, help detect abnormal transactions and potential market manipulation, and improve the accuracy of risk monitoring [30]. VAE, by learning the possible distribution of data, can generate realistic data samples that can be used to simulate different market scenarios, perform stress testing and risk assessment, and optimize risk management strategies.

3. The Application of Generative Artificial Intelligence in Financial Market Data Management

3.1. Data Collection and Preprocessing

1. Automate data collection and cleaning with generative AI

In financial markets, the diversity and quality of data directly affect the accuracy of forecasting models [31]. In addition, generative AI can also learn the underlying distribution of data, automatically fill in missing values, deal with abnormal data, and ensure data integrity and consistency.

2. Pretreatment techniques and tools

Data preprocessing is a critical step in data analysis and modeling, aiming to improve data quality, reduce noise, and enhance the performance of models. Standard preprocessing techniques include data cleaning, standardization, normalization, conversion, and feature selection [32]. In terms of tools, there are many powerful libraries in Python, such as Pandas for data processing, Scikit-learn for preprocessing and feature engineering, TensorFlow, and PyTorch for building and training deep learning models. These tools can help users efficiently preprocess data and provide a high-quality data foundation for subsequent modeling and analysis.

3.2. Data Enhancement and Synthesis

1. Use generative AI for data enhancement

Data enhancement is a method of transforming, extending, and generating new data samples from existing data to increase the diversity and quantity of data, thereby improving the robustness and generalization ability of the model. Generative AI, incredibly generative adversarial networks (GANs), and variational autoencoders (VAE) offer significant advantages in data enhancement [33]. Through these techniques, high-quality data samples with similar distributions can be generated, effectively expanding the size of the original data set. For example, Gans can generate stock price data under different market conditions in financial markets, helping models better capture market movements.

2. Examples of improving the diversity and quality of data sets

Here is an example of data enhancement using generative AI techniques:

Raw data set: historical trading data of a financial market, including stock prices, trading volume, etc.

Generative AI model: Gans are used to generate new samples and generate stock price and trading volume data under different market conditions by learning the distribution of the original data.

Data enhancement effect [34]: The generated data samples enrich the diversity of the original data set, cover more market fluctuations, and improve the model’s prediction ability in different market environments.

Table 1. Comparison Table Before and After Data Augmentation.

Dataset	Sample Size	Sample Diversity	Data Quality	Model Performance
Before	1000	Low	Medium	Average
After	3000	High	High	Excellent

Table 1. Comparison Table Before and After Data Augmentation.

Dataset	Sample Size	Sample Diversity	Data Quality	Model Performance
Before	1000	Low	Medium	Average
After	3000	High	High	Excellent

By leveraging generative AI for data augmentation, not only is the sample size significantly increased, but the diversity and quality of the data are also enhanced, thereby improving the performance and stability of predictive models.

4. Predictive Models of Financial Markets by Generative Artificial Intelligence

4.1. Construction of Prediction Model

Robust models must be designed and trained to use generative AI for financial market forecasting. Standard generative models include generative adversarial networks (GAN) and variational autoencoders (VAE) [35].

Model selection:

1. GAN: It is suitable for generating realistic financial time series data. The generative network generates realistic samples as much as possible, and the discriminant network analyzes whether the sample is accurate or generated as much as possible. GAN aims to obtain a generative network with good effect through this antagonistic game to be applied to image generation, speech generation, video generation, etc., such as the recently widespread “AI head change” may use GAN to generate acceptable “another face.” The specific indication of GAN is as follows:

Figure 3. GAN model.

Figure 3. GAN model.

Z is a hidden variable; random noise usually follows a Gaussian distribution. Z generates Xfake through the generator, and the discriminator distinguishes whether the input data generates sample Xfake or accurate sample Xreal. Model training aims to make the distributions of Xreal and Xfake as similar as possible. If measured by distance, the distance between the two distributions is the minimum. For example, cross-entropy can be expressed as:

(1)

2. VAE: Effective for learning potential representations and generating synthetic data. Unlike the traditional autoencoder, which describes the potential space numerically, it describes the potential space observation probabilistically, which shows excellent application value in data generation.

Figure 4. VAE model.

Figure 4. VAE model.

This paper uses a VAE model of a 3-layer encoder and decoder, and the activation function uses a GELU-Gaussian error linear unit. Finally [36], 112 features are generated, and then PCA is used to reduce the dimensions of these 112 features to explain 80% of the variance to form a high-dimensional feature combination. In the end, 84 feature combinations were produced.

4.2. Model Building

1. Generator

To analyze the sequence data, the generator naturally selects the extended short-term memory network LSTM (see the platform here for details). LSTM structure is relatively simple, features some input units, 500 hidden units, and one output unit (stock price), the loss function is L1 regularized mae, and the optimizer is adam.

LSTM principle: Take 17 days of data (these data are the stock price of GS stock for each day + all other features of the day - related assets, sentiment, etc.) and try to predict day 18. Then, move the window back, predict day 18 again, and iterate over the entire data set.

2. Discriminator

CNN is often used for image-related work. In stock price forecasting, data points form small trends, small trends form significant trends, and trends, in turn, form patterns, so CNN’s ability to detect features can be used to extract information about GS stock price trend patterns. This applies to time series, where the closer the two days are, the closer they are to each other [37]. One thing to consider is seasonality and how it might change what CNN does.

3. Hyperparameters

Parameter	Description
batch_size	The batch size of the LSTM and CNN
cnn_lr	The learning rate of the CNN
strides	The number of strides in the CNN
lrelu_alpha	The alpha for the LeakyReLU in the GAN
batchnorm_momentum	Momentum for the batch normalization in the CNN
padding	The padding in the CNN
kernel_size	The kernel size in the CNN
dropout	Dropout in the LSTM
filters	The initial number of filters

The stock market changes all the time. Even if GAN and LSTM are successfully trained to create accurate results, the results may only be valid for a certain period. In other words, the whole process needs to be continuously optimized:

1. Add or remove features (e.g., add new stocks or currencies that may be relevant)

2. Improve the deep learning model. One of the most important ways to improve the model is to optimize the hyperparameters

4.3. Model Results

When making predictions on the test set, the model effects of different times of epoch are as follows:

Figure 5. model result.

Figure 5. model result.

The experimental results conclude that the deep generation model shows excellent potential and practical application value in financial market prediction. We build a comprehensive deep-learning framework by integrating multiple data sources and feature extraction techniques, including fundamental analysis, technical indicators, global economic data, and sentiment analysis. After 200 times of epoch training and reinforcement learning optimization, the model significantly improved stock price prediction. In particular, the reinforcement learning method further enhances the model’s prediction ability through Q-learning and strategy optimization. It provides practical support for real-time market prediction and trading strategy.

These findings demonstrate the critical role of generative AI in financial market data management and forecasting and highlight its potential to handle complex market dynamics and improve forecasting accuracy. With the further evolution of technology and the richness of data, generative AI is expected to become a core tool for financial decision-making and investment management, bringing more significant benefits and innovation to the financial industry.

5. Conclusion

The experimental results of this study validate the effectiveness and practicality of generative AI in forecasting financial markets. By integrating GAN and VAE technologies, high-quality financial data samples are successfully generated, significantly enhancing forecasting models’ generalization ability and robustness across different market environments. Constructing a deep learning framework and applying reinforcement learning methods effectively reduce prediction errors in the market, providing reliable support for financial decision-making and investment management. These findings underscore the critical role of generative AI in financial data management and forecasting and offer new applications for future technological advancements and data richness. In light of ongoing developments, competition and cooperation within the international community on AI regulation are underway. In terms of competition, significant economies such as the United States and Europe are vying for influence in shaping global AI standards and regulatory rules; concerning cooperation, at the Hiroshima AI Conference in May 2023, G7 members agreed to develop a code of conduct to regulate the development and use of AI systems by significant companies. Furthermore, representatives from China, the United States, Europe, India, and other countries attended an Artificial Intelligence Summit held in early November, where they signed onto joint action plans to manage potential risks associated with artificial intelligence on a global scale while ensuring its safe development and responsible application. Given today’s complex international landscape marked by volatility, if Western powers dominate relevant standards, it may not be conducive to China’s AI industry development.