Grand Challenge on Detecting Cheapfakes

1University of Bergen, 2SimulaMet, 3NICT

Deepfakes (left): These are falsified media created using sophisticated AI-based media manipulation tools and techniques. Cheapfakes (right): These include falsified media created with/without contemporary non-AI based editing tools which are easily accessible. Photoshopping tools can be used to tamper with images. Videos can be sped up or slowed down to change the intent or misrepresent the person in the video. Re-contextualizing includes associating falsified or unrelated claims with a genuine image to misrepresent events or persons. This challenge is focused on detecting re-contextualized cheapfakes


Cheapfake is a recently coined term that encompasses non-AI ("cheap") manipulations of multimedia content. Cheapfakes are known to be more prevalent than deepfakes. Cheapfake media can be created using editing software for image/video manipulations, or even without using any software, by simply altering the context of an image/video by sharing the media alongside misleading claims. This alteration of context is referred to as out-of-context (OOC) misuse of media. OOC media is much harder to detect than fake media, since the images and videos are not tampered. In this challenge, we focus on detecting OOC images, and more specifically the misuse of real photographs with conflicting image captions in news items. The aim of this challenge is to develop and benchmark models that can be used to detect whether given samples (news image and associated captions) are OOC, based on the recently compiled COSMOS dataset.

Challenge Tasks

An image serves as evidence of the event described by a news caption. If two captions associated with an image are valid, then they should describe the same event. If they align with the same object(s) in the image, then they should be broadly conveying the same information. Based on these patterns, we define out-of-context (OOC) use of an image as presenting the image as an evidence of untrue and/or unrelated event(s)

Task 1

Every image in the dataset is accompanied by two related captions. If the two captions refer to same object(s) in the image, but are semantically different, i.e., associate the same subject to different events, this indicates out-of-context (OOC) use of the image. However, if the captions correspond to the same event, irrespective of the object(s) the captions describe, this is defined as not-out-of-context (NOOC) use of the image.

In this task, the participants are asked to come up with methods to detect conflicting image-caption triplets, which indicates miscontextualization. More specifically, given (Image,Caption1,Caption2) triplets as input, the proposed model should predict corresponding class labels (OOC or NOOC). The end goal for this task is not to identify which of the two captions is true/false, but rather to detect the existence of miscontextualization. This kind of a setup is considered particularly useful for assisting fact checkers, as highlighting conflicting image-caption triplets allows them to narrow down their search space.

Task 2

A NOOC scenario from Task 1 makes no conclusions regarding the veracity of the statements. In a practical scenario, multiple captions might not be available for a given image. In such a scenario, the task boils down to figuring out whether a given caption linked to the image is genuine or not. We argue that this is a challenging task, even for human moderators, without prior knowledge about the image origin. Luo et al. verified this claim with a study on human evaluators who were instructed not to use search engines, where the average human accuracy was around 65%.

In this task, the participants are asked to come up with methods to determine whether a given (Image,Caption) pair is genuine (real) or falsely generated (fake). Since our dataset only contains real, non-photoshopped images, it is suitable for a practical use case and challenging at the same time.

Paper Submission Guidelines

  • Paper Length: Papers must be no longer than 6 pages, including all text, figures, and references.
  • Format: Grand Challenge papers have the same format as regular papers. See the example paper under the General Information section below. However, their review is single blind.
  • Submission: Submit the written component via CMT under the appropriate Grand Challenge track. Submit the data component, if any, directly to the Grand Challenge organizers as specified on the appropriate Grand Challenge site.
  • Review: Submissions of both written and data components will be reviewed directly by the Grand Challenge organizers. Accepted submissions (written component only) will be included in the USB Proceedings and the authors will be given the opportunity to present their work at ICME. “Winning” submissions will be announced by the Grand Challenge organizers at the conference. Submissions may be accompanied by up to 20 MB of supplemental material following the same guidelines as regular and special session papers.
  • Presentation guarantee: As with accepted Regular and Special Session papers, accepted Grand Challenge papers must be registered by the author deadline and presented at the conference; otherwise they will not be included in IEEE Xplore. A Grand Challenge paper is covered by a full-conference registration only.

More details regarding submissions can be found here

Code Submission Guidelines

  • Participants can submit their solution as a notebook, e.g., Google Colab or Jupyter Notebook or Python Executable (.py file).
  • Please make sure that the notebook, or the .py file is directly executable by the challenge organizers without significant modifications.
  • Data injection should be possible by changing a single line to update the parameter INPUT_FOLDER. This value should be assumed to be the path to the folder containing the hidden test split file test.json.
  • Participants can upload their project (codes, models) to Google Drive, Microsoft Drive or any other similar online storage. The link must be shared with challenge organizers and also mentioned in the papers.
  • If the participants would like to share their solutions as Docker containers, please follow instructions here.
  • Participants can email the links to their Colabs, code repositories, or projects on Google Drive etc to Also, please make sure that the solution must accompanied by a report (word document, pdf) explaining the proposed solution, the achieved results, and other relevant details which can be helpful for the organizers to run and evaluate solutions.

Important Dates

Date Activity
February 12, 2023 Competition Starts
March 19, 2023
March 24, 2023
Deadline for submitting solutions and reports, marking the end of the competition.
March 23, 2023
March 29, 2023
Online publication of the final results, and invitations sent for submitting papers to the associated workshops at IEEE ICME 2023.
April 17, 2023 Deadline for submitting invited papers.
April 24, 2023 Notification of paper acceptance after reviews by the Competition Organizing Committee.
May 01, 2023 Deadline for camera-ready submission of accepted papers.


Team Task 1 - Public Task 1 - Private Task 2 - Public Task 2 - Private
Guangyang Wu et. al. 88.80% 72.27% - -
Thanh-Son Nguyen 87.60% 78.60% - -
Vinh Lã Tuấn 84.05% 76.36% 84.00% 67.50%

Out of all the submitted solutions, we were able to run three on our private test set. The results in the table above are not ranked yet. We will rank the results after reviews of the submitted papers become available to us.