This image was generated from a set of captured digital images used to train a Generative Adversarial Network, so would be classified as “trainedAlgorithmicMedia” in the proposed new version of the Digital Source Type CV. Source: Public Domain via Wikimedia Commons

A hot topic in media circles these days is “synthetic media”. That is, media that was created either partly or fully by a computer. Usually the term is used to describe content created either partly or wholly by AI algorithms.

IPTC’s Video Metadata Working Group has been looking at the topic recently and we concluded that it would be useful to have a way to describe exactly what type of content a particular media item is. Is it a raw, unmodified photograph, video or audio recording? Is it a collage of existing photos, or a mix of synthetic and captured content? Was it created using software trained on a set of sample images or videos, or is it purely created by an algorithm?

We have an existing vocabulary that suits some of this need: Digital Source Type. This vocabulary was originally created to be able to describe the way in which an image was scanned into a computer, but it also represented software-created images at a high level. So we set about expanding and modifying that vocabulary to cover more detail and more specific use cases.

It is important to note that we are only describing the way a media object has been created: we are not making any statements about the intent of the user (or the machine) in creating the content. So we deliberately don’t have a term “deepfake”, but we do have “trainedAlgorithmicMedia” which would be the term used to describe a piece of content that was created by an AI algorithm such as a Generative Adversarial Network (GAN).

Here are the terms we propose to include in the new version of the Digital Source Type vocabulary. (New terms and definition changes are marked in bold text. Existing terms are included in the list for clarity.)

Term ID digitalCapture
Term name Original digital capture sampled from real life
Term description The digital media is captured from a real-life source using a digital camera or digital recording device
Examples Digital photo or video taken using a digital SLR or smartphone camera
Term ID negativeFilm
Term name Digitised from a negative on film
Term description The digital media was digitised from a negative on film on any other transparent medium
Examples Film scanned from a moving image negative
Term ID positiveFilm
Term name Digitised from a positive on film
Term description The digital media was digitised from a positive on a transparency or any other transparent medium
Examples Digital photo scanned from a photographic positive
Term ID print
Term name Digitised from a print on non-transparent medium
Term description The digital image was digitised from an image printed on a non-transparent medium
Examples Digital photo scanned from a photographic print
Term ID humanEdited
Term name Original media with minor human edits
Term description Minor augmentation or correction by a human, such as a digitally-retouched photo used in a magazine
Examples Video camera recording, manipulated digitally by a human editor
Term ID compositeCapture
Term name Composite of captured elements
Term description Mix or composite of several elements that are all captures of real life
Examples * A composite image created by a digital artist in Photoshop based on several source images
* Edited sequence or composite of video shots
Term ID algorithmicallyEnhanced
Term name Algorithmically-enhanced media
Term description Minor augmentation or correction by algorithm
Examples A photo that has been digitally enhanced using a mechanism such as Google Photos’ “de-noise” feature
Term ID dataDrivenMedia
Term name Data-driven media
Term description Digital media representation of data via human programming or creativity
Examples A representation of a distant galaxy created by analysing the outputs of a deep-space telescope (as opposed to a regular camera)
An infographic created using a computer drawing tool such as Adobe Illustrator or AutoCAD
Term ID digitalArt
Term name Digital art
Term description Media created by a human using digital tools
Examples * A cartoon drawn by an artist into a digital tool using a digital pencil, a tablet and a drawing package such as Procreate or Affinity Designer
* A scene from a film/movie created using Computer Graphic Imagery (CGI)
* Electronic music composition using purely synthesised sounds
Term ID virtualRecording
Term name Virtual recording
Term description Live recording of virtual event based on synthetic and optionally captured elements
Examples * A recording of a computer-generated sequence, e.g. from a video game
* A recording of a Zoom meeting
Term ID compositeSynthetic
Term name Composite including synthetic elements
Term description Mix or composite of several elements, at least one of which is synthetic
Examples * Movie production using a combination of live-action and CGI content, e.g. using Unreal engine to generate backgrounds
* A capture of an augmented reality interaction with computer imagery superimposed on a camera video, e.g. someone playing Pokemon Go
Term ID trainedAlgorithmicMedia
Term name Trained algorithmic media
Term description Digital media created algorithmically using a model derived from sampled content
Examples * Image based on deep learning from a series of reference examples
* A “speech-to-speech” generated audio or “deepfake” video using a combination of a real actor and an AI model
* “Text-to-image” using a text input to feed an algorithm that creates a synthetic image
Term ID algorithmicMedia
Term name Algorithmic media
Term description Media created purely by an algorithm not based on any sampled training data, e.g. an image created by software using a mathematical formula
Examples * A purely computer-generated image such as a pattern of pixels generated mathematically e.g. a Mandelbrot set or fractal diagram
* A purely computer-generated moving image such as a pattern of pixels generated mathematically

We propose that the following term, which exists in the current DigitalSourceType CV, be retired:

Term ID RETIRE: softwareImage
Term name Created by software
Term description The digital image was created by computer software
Note We propose that trainedAlgorithmicMedia or algorithmnicMedia be used instead of this term.

 

We welcome all feedback from across the industry to these proposed terms.

Please contact Brendan Quinn, IPTC Managing Director at mdirector@iptc.org use the IPTC Contact Us form to send your feedback.