The Weaponisation of Artificial Intelligence (AI): Legal Shortfalls and Regulatory Difficulties in Governing Non-Consensual Intimate Deepfakes (NCIDs)

Artificial intelligence is a program that allows computers to exhibit human thought processes, such as learning, understanding, problem-solving, and creativity.10 Such programs can be utilised to create so-called deepfakes alongside a plethora of other uses. A deepfake is any non-genuine digital media created utilising deep learning technology; this can include images, videos and audio.11 Deepfakes can also include partially fabricated content;12 for example, a genuine image of an individual may be altered to make the individual look like they are in an intimate situation. The recently approved EU AI Act provides the first widespread legal definition of deepfakes: a deepfake is an ‘AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful.’13 The move to codify the definition of deepfakes indicates their rapidly increasing prevalence. In fact, in 2024, Ofcom reported that 43% of people aged 16+ had seen at least one deepfake in the last 6 months; concerningly, 50% of children aged 8-15 also reported having seen a deepfake within the same timeframe.14 More recently, joint research from the Alan Turing Institute and the University of Oxford propounded that 15% of people, on average, report exposure to harmful deepfakes, with 91.8% of respondents raising concerns regarding deepfake technology adding to online child sexual abuse material (CSAM), increasing disinformation and manipulating public opinion.15

Deepfakes can be used in a wide range of positive manners, benefiting society and individuals alike, for example, deepfakes have been used to help treat those with aphantasia (the inability to create mental images in one’s mind).16 However, as with everything in modern society, deepfakes can be abused and exploited by bad actors. An NCID is created when a perpetrator generates a deepfake of an individual, making them appear to be in an intimate situation without their consent. Criminals can use these NCIDs in various ways. For example, NCIDs may be used for sextortion,17 disrupting financial markets18 or creating CSAM.19

The rapid growth of intimate deepfakes online is astounding. Research by Deeptrace in 2019 identified that there were 14,678 deepfake pornography20 videos online, nearly a 100% increase from 2018.21 These videos have amounted 134,364,438 views across the top 4 websites alone; 96% of these deepfake videos online have been identified as being non-consensual.22 Our concerns have been echoed by both the previous Conservative and now Labour Governments.23 However, the legislation surrounding the ever-prevalent creation and distribution of NCIDs is poor, ineffective and in some cases non-existent; this will be discussed in detail in the coming chapters.

In addition to the prevalence of deepfakes and NCIDs there also comes difficulty when selecting appropriate terminology when discussing sensitive topics. The use of the term deepfake ‘porn’ is a frequently used colloquialism by the media,24 and even academics,25 when describing NCIDs. We opine the term ‘porn’ is misplaced and inappropriate. Rather, we submit that NCIDs appear to fall into the category of image-based sexual abuse defined as, ‘non-consensual creation and/or distribution of private sexual images,’ by McGlynn and Rackley.26 Terminology often sets debate frameworks and the routes for legal redress; therefore it is crucial to acknowledge that the creation and distribution of NCIDs is a form of image-based sexual abuse, not ‘deepfake porn’ or ‘revenge porn’.27 The use of the term ‘pornography’ is problematic, causing legislators to require images to be pornographic in nature or rooting culpability with a prerequisite that the images are generated for sexual gratification.28 This focus on the defendant’s mens rea is a stumbling block for prosecutions and securing closure for victims, this will be discussed in depth in the coming chapter. Additionally, McGlynn and Rackley opine that the monica ‘porn’ suggests the imagery is legitimate and ‘instils a sense of choice’; they submit this is inappropriate although note this does not suggest all pornography is abusive.29 To the contrary, others opine all pornography is abusive regardless of consent.30

To conclude, we submit that NCIDs are a form of image-based sexual abuse; as such, there is a necessity to stringently regulate this area to protect victim-survivors and hold offenders to account. We argue that due to the abusive nature and prevalence of NCID abuse, the government has a duty of care to the public, particularly women and girls, to introduce legislation that regulates the creation and distribution of NCIDs.

The psychological effects caused by the creation of NCIDs can have a significant effect on victims. Victims knowing their manipulated, sexualised image is being viewed for the sexual gratification of others without their consent can be profoundly disempowering,31 especially in the current climate where there are no options for redress until the image is shared. Research reveals that 57% of children are worried about becoming a victim of deepfake abuse, and 1 in 10 respondents reported being a victim of such abuse or knowing a victim.32 The impact of nonconsensual sharing of intimate images on adolescents is of particular notice. Research suggests that young victims of non-consensual creation and/or sharing of intimate images are at particular risk of developing severe psychological distress, in some cases developing depression and anxiety.33 Therefore, it is clear to us that the creation of NCIIs, including NCIDs, is incredibly harmful and has lasting psychological effects on victims. These effects can be elucidated by countless real-world examples.34

Due to rapid technological advancements, in some cases, NCIDs have become indistinguishable from genuine imagery.35 Therefore, when NCIDs are shared, the images may be difficult to tell apart from genuine images of the individual, which only increases the level of harm to the victim. These so-called ‘hyper-realistic’ images have become a stumbling block for law enforcement, for example, even expertly trained analysts from the Internet Watch Foundation (IWF) are sometimes unable to tell the difference between genuine and artificial CSAM, meaning they are unable to decide whether a child needs to be safeguarded.36 When it comes to investigating NCIDs, this could be problematic, with investigators not knowing if the images reported to them are genuine NCII or an NCID. This may present difficulties during investigations and with subsequent charging decisions.

In sum, it is clear that the creation and distribution of NCIDs can have a devastating and life-changing impact on a victim’s life. Again, we argue that NCIDs are a form of image-based sexual abuse as defined by McGlynn and Rackley and, therefore, should be treated as such.37 Next, we will turn to consider the current legal framework that governs NCIDs in the UK before considering our recommendations for reform.

Before the OSA, sections 33-35 of the Criminal Justice and Courts Act (CJCA) 2015 introduced offences relating to non-consensual disclosure of ‘private sexual photographs and films with the intent to cause distress’.41 However, Gillespie noted this provision was introduced solely to combat so-called ‘revenge porn’;42 significantly restricting its applicability to other, increasingly frequent, image-based sexual abuse. In addition, further concerns were lodged concerning the mens rea requirement of causing distress.43 This is an issue that will be discussed in detail in the coming chapter. Further, academic Kira highlighted the CJCA ‘did not directly apply to fully synthetic content’ and that s.34(6) specifically stated that a photograph or video was to be defined as, ‘a still or moving image that was originally captured by photography or filming, or is part of an image originally captured by photography or filming’.44 This stringently limits the CJCA’s applicability to NCIDs due to their artificial nature, leaving victim-survivors with little hope for legal redress or closure in cases where the imagery is totally artificially generated.

Communication offences may be applied when NCIDs are distributed using an electronic communications device. However, it is pertinent to note that these offences were not designed to combat such offending, with scholars McGlynn and Rackley arguing that the application of such provisions is haphazard at best.45 Section 1 of the Malicious Communications Act 1988 sets out the offence of sending a communication that is indecent, grossly offensive and threatening or false; however, there must be evidence of an intention to cause distress or anxiety to the victim.46 Taking the legislation at face value, it seems possible that s.1 could be used to prosecute an individual who shared an NCID since this could be considered ‘false’ as the imagery is not genuine.47 However, we must note that this would not have been the application of s.1 that legislators intended.

Additionally, section 127 of the Communications Act 2003 makes it an offence to send, using a public electronic communications network, a message that is ‘grossly offensive or of an indecent, obscene or menacing character’.48 However, this leaves ambiguity as to what would be considered grossly offensive, indecent, obscene or menacing. This was elucidated by the House of Commons when it was opined that s.1 and s.127 could not apply to non-consensual intimate imagery since the images are not necessarily grossly offensive; notably, Gillespie argued this is the incorrect element to focus on, and the ‘indecent’ or ‘obscene’ nature of the imagery was far more pertinent.49 We submit that the use of communications offences by the Crown to tackle NCIDs, whilst creative, is far from ideal, placing yet another hurdle between a victim-survivor and securing justice.

In April 2019, the government published the Online Harms White Paper50 before the first draft of the OSA was introduced as a Bill in May 2021.51 Following some parliamentary ‘ping-pong,’52 the bill received Royal Assent on 26th October 2023.

The OSA repealed the provisions mentioned above,53 introducing a range of criminal offences relating to image-based sexual abuse. The basic offence is established when person A intentionally shares an intimate photograph or film of person B without B’s consent or without a reasonable belief that B consented.54 Upon conviction, this base offence carries a maximum sentence of imprisonment for a term not exceeding the maximum term for summary offences or a fine (or both).55 In addition to this baseline offence, the OSA adds a series of more serious, aggravated forms of the s.66B(1) offence. These apply where person A shares an image, without consent, with the intention of causing B alarm, distress or humiliation,56 or A does so to secure sexual gratification for A or another individual.57 Finally, the OSA also tackles the issue of coercion and threatening behaviour. Here, an offence is committed if A threatens to share an intimate photograph or film and does so with the intention that B (or another) will fear the threat will be carried out or is reckless thereto.58 The increasing seriousness of these offences is demonstrated by the statutory maximum of (1) a custodial sentence not exceeding the limit in a magistrates court or a fine (or both) and (2) on conviction on indictment to a term of custody not exceeding 2 years.59 As submitted by Kira, the tiered system of penalties reflects the increasing degrees of harm and culpability associated with each offence.60 Critically, the provision adapts previous definitions of ‘photography’ and ‘film’ to include images ‘made or altered by computer graphics or in any other way, which appears to be a photograph or film’.61 Therefore, it seems the OSA has criminalised the distribution of NCIDs; however, it seems to have completely ignored the issue of creating an NCID in the first place. Therefore, it would be legal for an individual to create an NCID of a celebrity, family member or political figure so long as they did not distribute it, even if the NCID was generated for their own sexual gratification.

We opine that the OSA’s amendments to the SOA have significantly widened the scope of criminal liability, providing the Police and the Crown with comprehensive legislation to tackle IBSA, including deepfakes and other modified media. However, the OSA fails to criminalise the act of creating sexually explicit deepfakes and focuses far too much on the offender’s motives, elucidated by the requirement of the intention to cause alarm, distress or humiliation. The proof of motive requirement places yet another statutory hurdle in the way of holding offenders accountable and protecting victim-survivors. Additionally, the failure to criminalise the creation of NCIDs creates a market for its dissemination, we contend by criminalising the simple creation, we will see a reduction of NCID distribution offences in the future, ultimately protecting victims.

Now that we have considered the current legal frameworks in the UK that can be used to regulate NCIDs, it may be helpful to briefly consider other legal jurisdictions before we turn to our recommendations for reform. We argue the EU is the most suitable territory to examine due to the recent introduction of legislation designed to regulate the spheres of technology, particularly AI.63 Additionally, the EU provided the first widely accepted legal definition of deepfakes.64

In May 2024, the EU implemented a directive which aimed to harmonise legislation and policy in response to violence against women and girls across its Member States.65 The directive contains provisions relating to female genital mutilation,66 forced marriage,67 cyberstalking,68 cyber harassment,69 cyber violence70 and the non-consensual sharing of intimate or manipulated material.71

Article 5 will be the focus of our comparative analysis due to its focus on non-consensual intimate imagery, including the contention of artificially manipulated content. Article 5 criminalises the distribution of ‘materials depicting sexually explicit activities’ or ‘intimate parts’ where the person shown does not consent and the material is made public.72 Rigotti et al. opine that the EU’s legislative acknowledgement of this form of IBSA is a connotation of its prevalence and escalation in recent years.73

Article 5(1)(b) specifically ensures that artificially manipulated images and videos, known as deepfakes, are criminalised. It is an offence to distribute manipulated material of an individual that makes them look to be engaged in sexually explicit activities without their prior consent.74 However, article 5(1)(b) is only engaged if the individual appears to be ‘engaged in sexually explicit activities,’ but, confusingly, excludes images of nudity. Therefore, it would not be an offence under this directive to create a deepfake of an adult, making them appear naked, but not actively participating in a sexual act and distributing it without consent; even if the nude image could be considered sexual by itself. This gap in the law was identified; however, the commission took no action to amend the directive.75

We contend this is inexcusable. The use of so-called ‘nudify’ apps is increasingly prevalent and is harmful to victim-survivors,76 where naked imagery of them is shared without consent. For the law in the EU not to criminalise non-consensual sharing is unacceptable and leaves people, particularly women and girls, in an extremely vulnerable position. This places article 5(1)(b) significantly behind the development of the OSA in the UK, which criminalises the non-consensual sharing of intimate imagery without the hurdle of the imagery needing to depict sexual activity.77

Another significant limitation is that Article 5 is limited to the distribution or threats to distribute intimate imagery. Therefore, the creation of such non-consensual intimate imagery is not criminalised.78 This mirrors the current legislation in the UK, which also fails to criminalise the creation of NCIDs, instead choosing to focus on its distribution,79 or threats thereto.80 This is problematic since it allows individuals to generate NCIDs of colleagues, friends and family members for their sexual gratification without redress. We submit that this is morally contentious from the perspective of victims having their images digitally manipulated, sexualising them without their consent.81

In sum, the EU is at the forefront of developing legislation concerning artificial intelligence. The introduction of Article 5(1)(b) helpfully specifies that the directive does cover artificially manipulated content, known as deepfakes. So far, the EU has shown a level of commitment to combating gender-based sexual violence and includes technology-facilitated offences in this battle.82 However, as previously noted, it seems to be less developed than the OSA when it comes to criminalising the non-consensual sharing of intimate imagery. We will now turn our attention back to recent UK Government proposals from both the previous Conservative and present Labour governments relating to criminalising the creation of NCIDs.

In recent months, the issue of the lack of regulation surrounding the creation of NCIDs has been at the forefront of both media83 and government, with the Women’s Equality Committee publishing their enquiry on non-consensual intimate image abuse in March of 2025.84 This is an area of law that has developed rapidly throughout the research and writing of this project.

The government has pledged to introduce a provision which criminalises the creation of NCIDs.85 Under the planned offence, those who create NCIDs could face an unlimited fine and a criminal record; therefore, an individual creating an NCID with no intention to share it but instead wants to cause alarm, humiliation or distress to the victim will be committing an offence.86 This provision would be introduced through an amendment to the Criminal Justice Bill. However, when the 2024 general election was called by the then-Prime Minister Rishi Sunak MP, the proposals were dropped.87 As of the date of writing, the creation of NCIDs has not been criminalised in the UK.

A petition started by victim-survivor Jodie and #NotYourPorn demanding that the government implement laws to criminalise image-based abuse, including the creation of NCIDs, has received over 70,500 signatures.88

In January of 2025, the newly formed Labour government announced plans to continue their predecessors’ efforts to criminalise the creation of NCIDs.89 This is partly due to the advocacy efforts of End Violence Against Women Coalition, Jodie, #NotYourPorn, Glamour UK, Baroness Owen and expert Professor Clare McGlynn KC (Hon) of Durham University.

It is understood that the Government plans to make it an offence to create ‘artificial images either for ‘sexual gratification’ or to cause ‘alarm, distress, or humiliation’ as an amendment to the Data (Use and Access) Bill, which is currently being debated before parliament.90 This is similar to the conservative government’s initial plans but includes the aspect of sexual gratification rather than solely focusing on the intention to cause alarm, distress or humiliation. Scholars opine that the steps toward criminalising the creation of NCIDs are a ‘necessary and welcome step toward fostering individual accountability.’91

However, the Government’s plans, whilst generally receiving positive support, have attracted some criticisms due to their limited scope.92 To elucidate, there have been calls for the Government to introduce a specific offence to cover the solicitation of NCIDs; a call that had been initially rejected.93 However, since then, the Government has made a U-turn,94 intending for the law to include asking someone else to create an NCID for you, known as a solicitation.95 Here, Professor McGlynn’s evidential submissions96 were noted as being ‘overwhelmingly persuasive’.97 We conclude that this is a step in the right direction, preventing offenders from avoiding liability for the creation of NCIDs by asking someone outside of the UK to create it for them.

In addition to the issue of solicitation, similar concerns to those raised concerning the CJCA may also be pertinent here. Scholars were concerned by the mens rea focus, namely the need to prove the intention to cause distress.98 The government plans to introduce an offence which again relies on the intention to cause ‘alarm, distress, or humiliation’ although we must note the addition of ‘sexual gratification’.99 We submit that the overreliance on the defendants’ motives and intentions will be problematic and will likely cause prosecutorial delays with defendants avoiding liability due to the difficulty in proving the defendant’s intentions. For example, if a defendant created an NCID of an individual but argues it is simply a hobby or a piece of art, it would be increasingly difficult to prove otherwise. Unfortunately, this places vulnerable victims in a difficult situation due to the drafting of the planned offence. We opine that the government should amend the creation offence to criminalise the non-consensual creation of intimate deepfakes, a point we will discuss in detail in the next chapter.

To conclude, we suggest the labour government’s inclusion of sexual gratification as an additional mens rea element is a step in the right direction, which widens the situations in which the mens rea requirement can be satisfied. In addition to this, the government’s change in opinion on the matter of socialisation is welcomed; again, this strengthens the planned legislation, ensuring individuals can not escape liability by simply asking someone in a different territory to generate an NCID for them. However, whilst the proposed legislation has developed significantly, it remains incomplete. We opine that instead of an offence which hinges on the defendant’s mens rea the government should instead introduce a consent-based model. This would make the creation of NCIDs prohibited if the person depicted did not consent. In the next chapter, we will introduce our fully synthesised recommendation for the introduction of a consent-focused offence for the creation of NCIDs.

As of writing, the law in the UK criminalises the distribution of an ‘intimate photograph or film’ where the individual pictured does not consent.103 Helpfully, the OSA has adopted the definition of a photograph to include images ‘made or altered by computer graphics or in any other way, which appears to be a photograph or film’.104 Therefore, it is clear that s.66B(1) criminalises the non-consensual distribution of an NCID. However, the law does not, at present, criminalise the creation of NCIDs. Therefore, an individual can legally generate non-consensual sexual imagery of another, even if it is intended to be used maliciously in the future.

We recommend that the Government introduce a standalone offence that criminalises the creation of NCIDs. There is already similar legislation designed to prevent the creation of AI-generated CSAM.105 Therefore, we can conclude that this type of legislation is a feasible recommendation. We opine this would reduce the criminalised sharing of NCIDs by deterring the production of such imagery at its source, ultimately protecting the public.

We reject the argument of the Law Commission that harm only arises when an NCID is shared or is threatened to be shared.106 Rather, we agree that the creation of NCIDs is a form of sexual violence, and permitting such behaviour legitimises individuals intruding on the privacy and sexual autonomy of another.107 Additionally, we share the view that deepfake abuse is a major contributor to the systematic degradation and oppression of women.108

When discussing criminalising the creation of NCIDs, a common response is that deepfakes are an expression of an individual’s sexual fantasy and are no different to imagining it in one’s head.109 However, we submit this is not the case; it is instead creating a digital file that could be shared at any moment, even accidentally or via malicious routes such as unauthorised access.110 Therefore, the level of harm and culpability can not be compared to mere imagination; the creation of an NCID is simultaneously morally contentious and distinctly wrong.111

We submit that the Law Commission’s recommendation that it should not be a criminal offence to ‘make’ an intimate image without consent is categorically incorrect and represents a level of ignorance when it comes to sexual violence and abuse.112

Current and planned legislation to combat the creation and distribution of NCIDs hinges on the individual’s intention to cause ‘alarm, distress, or humiliation’ or to gain ‘sexual gratification.’113 However, the drafters seem to have completely ignored the issue of consent.

The concept of consent114 is central to defining sexual offending; without consent, the construction of offences such as rape and sexual assault would be impossible.115 Legally recognised consent is what separates an action of intimacy between partners from a serious sexual offence and a violation of the human dignity and personal autonomy of one of those people.116 Creating an NCID should be no different. Recent victims of NCID abuse have spoken about the ‘life-shattering’ effects NCID abuse can have.117 In addition to this, researchers have demonstrated the relationship between non-consensual sexual imagery dissemination and depression and low self-esteem in adolescents.118 Consequently, we opine that non-contact, online offending, such as the creation and distribution of NCIDs, can be just as harmful as other types of sexual offending, which hinge on the issue of consent.

To conclude, we argue that the offence of creating NCIDs should be consent-based rather than based on the defendant’s mens rea; this would mitigate the risk of defendants avoiding liability by arguing they did not intend to cause ‘alarm, distress, or humiliation’ and claiming they did not gain ‘sexual gratification.’119 Instead, it would focus on the lack of consent of the person depicted. This is the accepted standard of liability in the vast majority of sexual offences; NCIDs are a form of image-based sexual abuse and should be treated no differently.120