Enhancing Cloud Security with AI-Driven Big Data Analytics

The American Journal of Engineering and Technology

185

https://www.theamericanjournals.com/index.php/tajet

TYPE

Original Research

PAGE NO.

185-191

DOI

10.37547/tajet/Volume07Issue05-18

OPEN ACCESS

SUBMITED

24 March 2025

ACCEPTED

22 April 2025

PUBLISHED

28 May 2025

VOLUME

Vol.07 Issue 05 2025

CITATION

Vijaya lakshmi Middae. (2025). Enhancing Cloud Security with AI-Driven
Big Data Analytics. The American Journal of Engineering and Technology,
7(05), 185

–

191. https://doi.org/10.37547/tajet/Volume07Issue05-18

COPYRIGHT

© 2025 Original content from this work may be used under the terms
of the creative commons attributes 4.0 License.

Enhancing Cloud Security
with AI-Driven Big Data
Analytics

Vijaya lakshmi Middae

Dept of Computer and Information Sciences Memphis, TN, USA

Email ID - srilakshmio1a@gmail.com

Abstract:

Since cloud computing is changing so rapidly,

maintaining strong security is now a major issue for
companies everywhere. Massive volumes of mixed data
are constantly created in cloud environments at every
layer, involving virtual machines, containers, storage,
identity management and application activities. It is
usually not possible for traditional security systems and
old monitoring tools to manage vast and changing data
flow in real time. Con- ventional methods fail to discover
advanced persistent threats, attacks by team members
and new vulnerabilities because they do not easily
adjust to changing situations. To fix the urgent problem
of weak security in cloud sys- tems, this research
introduces an AI-powered big data analytics system. The
aim is to use artificial intelligence and big data
technologies to improve spot- ting threats, marking
unusual incidents and reducing risks as they happen.
Machine learning and deep learning are used within the
system which makes use of distributed processing
platforms such as Apache Spark, Hadoop and Kafka.
Together, these pieces ensure that a lot of log data and
telemetry from hybrid and multi-cloud setups are
ingested, worked on and analyzed quickly and
efficiently. The proposed solution uses Isolation Forests,
Ran- dom Forests, Autoencoders and LSTM networks to
spot abnormal activity and risks. They can recognize
unusual patterns in network activity, website logs and
API usage to find out about possible attacks. It also
makes use of natural language processing to study
unstructured log data for threats and compares these to
the ones listed in external threat intelligence. The archi-
tecture is built with a layer using Kafka and Logstash to
get data ingested, another using Spark and HDFS for
processing and a third for real-time threat analysis and
prediction with AI. Information about threats is

The American Journal of Engineering and Technology

186

https://www.theamericanjournals.com/index.php/tajet

presented vi- sually in dashboards with the help of
Grafana and Kibana, so analysts can easily respond to
any threats. Risks are scored with a mechanism that
focuses on the worst incidents and those expected to
have the biggest impact. Bench- mark datasets such as
CICIDS 2017 and UNSW-NB15 are used, along with
anonymized real-world activity logs from the cloud, to
assess the suggest

ed solution’s robustness. The data

suggests that using this technology is more effective and
faster than using traditional security approaches. This
study has resulted in an AI-based security framework
that can handle large enter- prise loads, adaptive
security models and affordable implementation paths
for the cloud. Thanks to this work, cloud security can
now focus on ad- vancing to automating early detection,
providing continuous monitoring and implementing
automatic steps when needed. Ultimately, the use of AI
and big data analytics changes how cloud security
functions. This research en- ables systems to detect
threats and rate risks in real time, helping to improve the

security of today’s cloud networks.

Keywords:

Cloud security, big data analytics, artificial

intelligence, real-time threat detection, anomaly
detection, machine learning, deep learning, cyber threat
intelligence, security analytics, Spark streaming, cloud
log analysis, predic- tive security.

Introduction:

The quick shift to cloud computing has

deeply changed how companies han- dle their data.
Enterprises prefer cloud systems for their ability to
adjust, scale and reduce costs. Since cloud platforms
such as AWS, Azure and GCP can handle mission-
important applications and big data requirements, they

are key parts of today’s digital environments. The

increase in cloud use has created new issues related to
data security. Issues include configuration mis- takes,
unapproved access, data leaks, threats from staff
members and stable attacks that target cloud
weaknesses. The rules, monitoring and signature
schemes in traditional security have problems coping
with the changing and flexible environment of the cloud.
Because of this, old systems are not able to deal with the

high volume, speed and intricacy of today’s cloud

workloads, putting them at risk from new threats that
appear quickly. In addition, be- cause cloud platforms
generate lots of log and telemetry data quickly and in
many forms, it becomes difficult to find security
incidents or respond to them using standard

approaches. To overcome these limitations, people are
increasingly trying to add AI and big data analytics to
cloud security, and it helps by allowing organizations to
predict risks, use intelligent tools and adjust to new
problems in real time.

The goal of this research is to build and use an AI and big
data framework that detects threats, finds unusual
activities and automatically addresses risks. Machine
learning and deep learning are used in the framework to
review large amounts of security data pulled from
different cloud identities, networks, web API activity and
system-related events. Through Apache Spark, Kafka
and Hadoop, the system can process large, assorted
data in real time without overloading. The plan explains
how data will be recorded through security logs, then
processed for cleaning by another layer, followed by
classification and prediction from a third AI-supported
layer. A new way of scoring threats by how severe they
are is introduced so different threats can be managed
faster. Using visualization dashboards, it becomes easier
for security teams to see what the security posture and
threat landscape look like. This research helps progress
towards a flexible security approach for cloud
computing by acknowledging challenges in traditional
security tools and introducing AI in data analysis. The
plan is to help organizations spot and deal with
problems beforehand, cut down on false alarms and

strengthen their defenses in today’s challengi

ng cyber

scenario.

Literature Review

1.

Evolution of Cloud Computing and Its Security

Chal- lenges

Earlier, items such as infrastructure and services had to
be bought by orga- nizations. Now, cloud computing
enables users to access services on demand. However,
moving to cloud environments introduces a broader
attack surface, increasing exposure to cyber threats.
Traditional tools struggle to manage decentralized
architectures, dynamic workloads, and multi-tenant
environ- ments. Key issues include data breaches,
unsecured APIs, account hijacking, misconfigured
storage, and insider threats. The shared responsibility
model between users and cloud providers further
complicates security, increasing de- mand for intelligent,
automated, and scalable solutions that adapt to evolving
threats

The American Journal of Engineering and Technology

187

https://www.theamericanjournals.com/index.php/tajet

2.

Limitations of Traditional Security Approaches

Conventional security methods, such as intrusion
detection systems (IDS) and security information and
event management (SIEM), often rely on static
signatures and manually defined rules. While effective
against known threats, they are insufficient for detecting
unknown or sophisticated attacks. High false-positive
rates burden analysts and delay incident response.
Further- more, the vast volume, velocity, and variety of
cloud data reduce their effec- tiveness in real-time
threat detection.

3.

The Role of Big Data Analytics in Cloud Security

Big data analytics enables organizations to ingest,
process, and analyze mas- sive volumes of
heterogeneous cloud data. Scalable platforms like
Apache Hadoop, Spark, and Kafka facilitate real-time
streaming and batch processing of logs, telemetry,
events, and network traffic. These tools support
anomaly detection, forensic analysis, and threat hunting
by storing and querying his- torical data. Through
advanced analytics, unusual behavior patterns can be
identified, improving overall cloud security posture.

4.

Application of Artificial Intelligence in Threat

De- tection

AI

—

especially machine learning (ML) and deep learning

(DL)

—

is becom- ing essential in cybersecurity. ML

models learn from historical security data to
differentiate between normal and abnormal behaviors.
Techniques such as Random Forest and Support Vector
Machines (SVM) handle supervised classification, while
unsupervised methods like Isolation Forests and cluster-
ing reveal novel threats. Deep models like autoencoders
and LSTM networks are suitable for sequential data like
log files and network flows, allowing de- tection of
subtle or evolving attack patterns. AI is also being
explored for real-time decision-making and autonomous
response.

5.

Integration of AI and Big Data for Cloud

Security

Combining AI with big data in cloud environments
fosters proactive and predictive security. Streaming
platforms like Spark Streaming and Apache Flink enable
real-time ingestion and processing of cloud logs.

Simultane- ously, AI models detect anomalies, such as
unauthorized access or data ex- filtration. Visualization
tools like Kibana and Grafana enhance situational
awareness by presenting insights in intuitive
dashboards. This integration enables self-adaptive
systems that reduce false alarms and improve threat
detection accuracy.

6.

Recent Research and Industrial Applications

Contemporary research supports combining anomaly
detection, natural lan- guage processing (NLP), and
ensemble learning for robust threat manage- ment.
Cloud providers like Microsoft Azure and Amazon Web
Services (AWS) have introduced AI-enhanced security
services

—

Azure Sentinel and AWS GuardDuty

—

capable

of identifying threats, applying updates, and triggering
automated responses. However, these proprietary
systems lack transparency and customizability,
underlining the need for open, research- supported
security frameworks.

7.

Gaps and Research Opportunities

Despite advances, challenges remain. AI models in cloud
security often face issues related to scalability, high
resource costs, and lack of explainability. Adversaries
can exploit models through data poisoning and evasion
tech- niques. Additionally, the scarcity of realistic, cloud-
native attack datasets limits the practical evaluation of
AI solutions. Existing academic datasets fail to capture
the complexities of real-world environments. More
collabora- tion with cloud providers, development of
robust testing environments, and attention to fairness,
privacy, and regulatory compliance are necessary for
progress.

8.

Summary of Literature Insights

The literature highlights the urgent need for intelligent,
adaptable, and scalable cloud security systems. AI and
big data analytics enhance situa- tional awareness,
enable automated detection, and reduce incident
response times. However, challenges in model
interpretability, resistance to adversar- ial threats, data
availability, and real-world deployment must be
addressed. This research aims to bridge these gaps by
proposing a real-time, AI-powered big data analytics
framework tailored for secure cloud environments.

The American Journal of Engineering and Technology

188

https://www.theamericanjournals.com/index.php/tajet

Figure 1: AI in logistics.jpg

3

Future Scope of Enhancing Cloud Security with

AI-Driven Big Data Analytics

The future of cloud computing hinges on securing vast,
distributed envi- ronments that operate at scale and
speed. With cyber threats growing in sophistication and
frequency, the integration of AI-driven big data analytics
into cloud security systems is not just an innovation

—

it

is a necessity. This section outlines the key directions
and emerging opportunities shaping the future of this
research area.

3.1

Autonomous and Self-Healing Security Systems

One significant area of advancement is the development
of autonomous secu- rity architectures capable of self-
monitoring, self-diagnosis, and self-repair. These
systems will leverage reinforcement learning and
adaptive AI algo- rithms to dynamically respond to
evolving threats. Such platforms can de- tect anomalies,
isolate compromised resources, automatically update
config- urations, and restore normal operations with
minimal human intervention.

Over time, they will continuously learn from new
attacks, enhancing their defensive strategies.

3.2

Federated and Privacy-Preserving Learning

With growing concerns around data privacy and
regulatory compliance, fed- erated learning has
emerged as a promising approach. This method enables
the training of AI models across decentralized data
sources without transfer- ring raw data to a central

server. In cloud environments, federated learning allows
organizations

—

especially in sectors like healthcare,

finance, and gov- ernment

—

to collaboratively develop

threat detection models while preserving data
confidentiality and adhering to strict compliance
requirements.

3.3

Explainable and Trustworthy AI in Cloud

Security

As AI becomes increasingly integral to cloud security
operations, explain- ability and transparency are critical.
Future AI frameworks must incorpo- rate explainable AI
(XAI) to ensure decisions are interpretable by security
analysts, auditors, and regulators. These systems will
provide insights into why specific actions were taken,
how anomalies were identified, and which features
influenced model outputs, thereby fostering trust and
enabling ac- countability.

3.4

Real-Time, Predictive, and Proactive Defense

Mech- anisms

The future of cloud security will increasingly rely on
predictive analytics to identify threats before they
materialize. By analyzing historical data, behav- ioral
patterns, and threat intelligence, machine learning
models can forecast high-risk assets, likely attack
vectors, and propagation paths. When inte- grated with
real-time big data platforms such as Apache Kafka and
Apache Flink, these models will enable early threat
detection and proactive incident response.

The American Journal of Engineering and Technology

189

https://www.theamericanjournals.com/index.php/tajet

3.5

Integration of Blockchain for Trust and

Integrity

Blockchain

technology

offers

immutable

and

decentralized record-keeping, making it highly valuable
for ensuring data integrity and traceability in cloud
security. Integrating blockchain with AI can enhance the
credibility of security logs, facilitate forensic analysis,
and prevent tampering by attack- ers. Securing audit
trails on blockchain infrastructures will lead to increased
transparency, accountability, and trust in cloud-based
security operations.

3.6

Quantum-Resistant AI for Post-Quantum Cloud

Environments

The advent of quantum computing poses significant
threats to classical cryp- tographic protocols and AI

models. Future cloud security systems must be designed
with quantum resilience in mind. This includes adopting
quantum- resistant algorithms such as lattice-based
cryptography and hash-based sig- natures. Moreover,
protecting AI workflows

—

training, inference, and com-

munication

—

against

quantum-enabled

adversarial

attacks will be a critical research focus.

3.7

Unified AI and SIEM/SOAR Integration

Security Information and Event Management (SIEM) and
Security Orches- tration, Automation, and Response
(SOAR) platforms will undergo a trans- formation
through advanced AI integration. AI-driven systems will
replace traditional rule-based engines, enabling
automated event correlation, prior- itization, root cause
analysis, and incident response. This unified approach
will streamline operations and improve threat visibility
across hybrid and multi-cloud environments.

3.8

Intelligent Cloud Compliance and Governance

Au- tomation

Ensuring compliance in multi-cloud environments
remains a complex chal- lenge due to evolving regional,
industry, and legal requirements. AI and big data
analytics will automate compliance monitoring and
reporting, identi- fying policy violations and generating
audit-ready

documentation.

Machine

learning

algorithms will map operational activities to standards
such as ISO 27001, NIST, HIPAA, and PCI DSS, allowing
organizations to maintain con- tinuous compliance and
reduce manual overhead.

4

CONCLUSIONS

With cloud environments getting larger, more
complicated and more impor- tant, they need to be
protected with more than standard static approaches. It
was discussed in this paper how AI-embedded data
processing for big data evidence in cloud security can
actively and intelligently respond to new cybersecurity
issues. Easily analyzing a lot of current data, AI
algorithms find regularities, identify anomalies and
predict future risks to security which helps respond and
deal with them as soon as possible.

Because artificial intelligence and big data analytics
collaborate, cloud security systems turn into automated,
learning systems that quickly respond and protect
against emerging threats. Thanks to federated learning,
ex- plainable AI and blockchain integration, there is
greater data privacy, trans- parency and integrity in the
system. Introducing autonomous defenses and post-
quantum security enhancements will largely increase
the durability of upcoming cloud infrastructure.

As a result, cybersecurity is now evolving so that cloud
systems can ad- just, defend and repair problems mostly
on their own. Using advanced secu- rity barriers bolsters
cloud systems and also reassures users, enterprises and
regulators.

As a result, AI-based big data analytics will help build the
future of cloud protection, ensuring businesses can
handle threats while keeping up their growth, following
regulations and reducing costs.

Despite these advancements, challenges remain, such as
data quality is- sues, computational resource demands,
and the need for more interpretable AI models.
Overcoming these obstacles will require continuous
research, improved AI transparency, and the seamless
integration of AI into existing logistics frameworks.
Nevertheless, as AI continues to evolve, demand fore-
casting will become increasingly accurate, intelligent,
and essential for busi- nesses striving to maintain a
competitive edge in a rapidly changing market. The
future of logistics will be shaped by AI-driven insights,
leading to more resilient, cost-effective, and agile supply
chain systems.

REFERENCES

[1]

M. A. Ferrag, L. Maglaras, A. Derhab, and H.

The American Journal of Engineering and Technology

190

https://www.theamericanjournals.com/index.php/tajet

Janicke, “A review of the security of distributed ledger
technologies,” IEEE Trans. Services Comput., vol. 13, no.

3, pp. 550

–

563, 2020.

[2]

Y. Yuan and F.-

Y. Wang, “Blockchain: The state

of the art and future trends,

” Acta Automatica Sinica,

vol. 45, no. 4, pp. 217

–

223, 2018.

[3]

N. A. B. Solochie et al., “Machine learning for

cybersecurity: A comprehensive survey,” IEEE Access,

vol. 11, pp. 32461

–

32488, 2023.

[4]

A. K. Abeshu and N. Chilamkurti, “Deep learning:

The frontier for distributed attack detection in fog-to-

things computing,” IEEE Commun. Mag., vol. 56, no. 2,

pp. 169

–

175, 2018.

[5]

S. A. Shinde and S. A. Khatoon, “Privacy

-

preserving federated learning for healthcare systems,”

IEEE J. Biomed. Health Inform., vol. 25, no. 4, pp. 1335

–

1342, 2021.

[6]

S. R. Pokhrel and J. Choi, “Federated learning

with blockchain for autonomous vehicles: Analysis and

design challenges,” IEEE Trans. Commun., vol. 68, no. 8,

pp. 4734

–

4746, 2020.

[7]

I. Goodfellow, Y. Bengio, and A. Courville, Deep

Learning. MIT Press, 2016.

[8]

A. M. Tjoa and S. Tjoa, “Cybersecurity in the era

of AI: Current challenges and future research

directions,” in Proc. 2018 Int. Conf. Cyber Security Prot.

Digital Services, 2018, pp. 1

–

8.

[9]

S. A. Shaikh and

S. A. Khatoon, “AI and ML in

cloud security: Challenges and opportunities,” J. Cloud

Comput., vol. 9, no. 1, pp. 1

–

20, 2020.

[10]

P. K. Sharma, S. Rathore, and J. H. Park,

“DistBlockNet: A dis

- tributed blockchains-based secure

SDN architecture for IoT net-

works,” IEEE Commun.

Mag., vol. 55, no. 9, pp. 78

–

85, 2017.

[11]

National Institute of Standards and Technology,

“Post

-

Quantum Cryptography: NIST’s Plan for the

Future,”

2022.

[Online].

Avail

-

able:

https://csrc.nist.gov/

[12]

M. Al-Rubaie

and J. M. Chang, “Privacy

-

preserving machine learn-

ing: Threats and solutions,”

IEEE Secur. Privacy, vol. 17, no. 2,

pp. 49

–

58, 2019.

[13]

M. Kantarcioglu and B. Xi, “Adversarial machine

learning in cyber-

security,” in Proc. IEEE 16th Int. Conf.

Data Mining Workshops, 2016, pp. 1305

–

1310.

[14]

R. Shokri and V. Shmatikov, “Privacy

-preserving

deep learning,” in Proc. 22nd ACM SIGSAC Conf.

Computer Commun. Security, 2015, pp. 1310

–

1321.

[15]

M. S. Ali, M. Vecchio, M. Pincheira, K. Dolui, F.

Antonelli, and

M

. H. Rehmani, “Applications of blockchain in the

Internet of Things: A comprehensive survey,” IEEE

Commun. Surveys Tu- torials, vol. 21, no. 2, pp. 1676

–

1717, 2019.

[16]

T. T. Nguyen and G. Armitage, “A survey of

techniques for inter- net traffic classification using

machine learning,” IEEE Commun. Surveys Tutorials, vol.

10, no. 4, pp. 56

–

76, 2008.

[17]

A. M. Lone and R. Naaz, “Quantum

-safe

cryptography: A survey,”

J. Inf. Secur. Appl., vol. 61, p. 102925, 2021.

[18]

A. Jain and D. Singh, “AI

-powered threat

detection for cloud en- vironments: A real-time analytics

approach,” J. Cloud Comput., vol. 12, no. 1, pp. 1–

15,

2023.

[19]

B. Tang and Q. Zhang, “Big data analytics in

cloud computing: A survey,” Future Gener. Comput.

Syst., vol. 37, pp. 209

–

220, 2014.

[20]

A. Madakam, S. Ramaswamy, and R. Tripathi,

“Internet of Things (IoT): A literature review,” J. Comput.

Commun., vol. 3, no. 5, pp. 164

–

173, 2015.

[21]

S. Yerra, “Reducing ETL processing time with

SSIS optimizations for large-

scale data pipelines,” 202

5.

[Online]. Available: https:

//doi.org/10.55640/ijdsml-05-01-12

[22]

S. Yerra, “Optimizing supply chain efficiency

using AI-driven pre-

dictive analytics in logistics,” 2025.

[Online].

Available:

https://

ijsrcseit.com/index.php/home/article/view/CSEIT25112
475

[23]

S. Yerra, “Enhancing inventory management

through real-time Power BI dashboards and KPI

tracking,”

2025.

[Online].

Avail

-

able:

https://ijsrcseit.com/index.php/home/article/view/

The American Journal of Engineering and Technology

191

https://www.theamericanjournals.com/index.php/tajet

CSEIT25112458

[24]

S. Yerra, “Leveraging Azure DevOps for backlog

management and sprint planning in supply chain,”

Journal of Information Systems Engineering and
Management, vol. 10, no. 36, pp. f1019

–

f1023, 2025.

[Online]. Available: https://jisem-journal.com/index.
php/journal/article/view/6629

[25]

S. Yerra and V. L.

Middae, “Intelligent workload

readjustment of serverless functions in cloud to edge

environment,” International Journal of Data Science and

Machine Learning, 2025. [Online].

Available: https://doi.org/10.55640/ijdsml-05-01-18

[26]

S. Talwar,

“Dynamic Just

-In-Time app

servers with auto-

mated access management on AWS,”

2025.

[Online].

Available:

https://computerfraudsecurity.com/index.php/journal/
article/view/411/280