CodeFormer

CodeFormer restores degraded face images using a learned visual prior and a Transformer. It can produce a more legible portrait, but newly generated facial detail is not evidence of what was present in the original scene. Evaluate it for a permitted display copy when you can inspect changes and preserve the source. The official project uses S-Lab License 1.0, which permits specified non-commercial uses and directs commercial users to contact the contributors.

This is a source-based evaluation of the official restoration implementation and research, not a report of image tests. No model inference, timing comparison or hosted-service submission was performed for this page. The family-photo exercise below is a proposed evaluation with unfilled results.

Decide what the restored portrait needs to preserve

The research frames blind face restoration as an ambiguous reconstruction problem. CodeFormer uses a learned discrete codebook as a visual prior, with a Transformer predicting code sequences. This helps explain both its intended use and its limit: a plausible face can contain inferred features that the damaged input does not establish. The paper is by Shangchen Zhou, Kelvin C. K. Chan, Chongyi Li and Chen Change Loy, affiliated with S-Lab at Nanyang Technological University. CodeFormer research paper.

Consider a scanned family portrait with compression artifacts, soft eyes and a scratch crossing one cheek. The reader's job is to make a display derivative that remains acceptable to the people represented. Before changing the image, identify what is actually visible: head angle, mouth shape, expression, distinctive marks and the relationship between the face and hair. Mark uncertain details as uncertain. A crisp eyelid or tooth added later does not resolve that uncertainty.

Keep the scan as the archival source and give the edited image a separate filename. Decide whether a slightly soft but recognisable face serves the purpose better than a sharper face with changed expression. This is an editorial acceptance decision, not a score produced by the model. Someone familiar with the person may help notice changes, but agreement with a familiar appearance still does not make generated detail a historical record.

Do not use this workflow to establish an unknown person's identity, reconstruct evidentiary detail or certify that a face is authentic. When those facts are the task, improving visual plausibility is not an adequate success criterion. A better scan, another permitted reference or retaining the damaged source may be the appropriate outcome.

Separate a face crop from the whole-image workflow

The official restoration script has two relevant entry paths. With --has_aligned, it treats the input as an aligned face and resizes it to 512 × 512. Without that flag, it detects faces, aligns face crops, restores them and can paste them back into the image. Its default detector is retinaface_resnet50; --only_center_face changes face selection. The script and its arguments, rather than a wrapper's product description, define this reviewed scope. Official inference implementation.

Choose the entry path from the material you actually have. An ordinary portrait is not necessarily an aligned face crop. In a family group, selecting only the centre face can exclude someone important. Inspect the selected crops before deciding that the restoration model missed a person or changed their face. If the wrong face reaches the restoration stage, adjusting the fidelity weight cannot correct that selection.

Whole-image quality includes recomposition. An acceptable isolated face may look pasted on when its sharpness, colour or boundary differs from the surrounding photograph. Review the returned face both at the crop's working scale and inside the full image at the intended display size. A tightly cropped comparison can conceal a seam near the hairline or an unnatural transition to the neck.

Optional Real-ESRGAN processing is a separate upsampling component in the official workflow. Background upsampling and face upsampling are separate switches; they are not prerequisites for every restoration. Begin without adding these stages so you can attribute a change to the relevant operation. The repository's aligned-face and whole-image examples have different purposes, so their outputs should not be treated as interchangeable evaluation conditions. Repository usage guidance.

Compare fidelity settings against the same source

The documented fidelity weight runs from 0 to 1. The repository describes smaller values as favouring visual quality and larger values as favouring fidelity. This is a control to investigate on your input, not a universal ordering of better and worse portraits. The CLI defaults to 0.5, which is a starting configuration rather than a recommendation for every photograph.

Hold the input, face-selection mode and other processing settings fixed while comparing weights. Use separate output folders and record the value with each image. Avoid selecting a result solely because its eyes look sharper: compare the same eyes, mouth and characteristic features with the original. A setting that works for one face in a group may be unacceptable for another.

Setting the weight to 1 does not promise an unchanged copy of the input. The architecture still performs learned restoration; its fusion path uses the weight to control features. Preserve the original independently instead of relying on one endpoint of the slider as an archival safeguard. CodeFormer architecture.

If every setting changes an important feature, reject those derivatives. Repeatedly adjusting a slider cannot demonstrate that a missing feature has been recovered. Record the rejection reason so that a later upsampling or retouching stage does not quietly reintroduce the same unacceptable face.

Check the actual license and access path

The official repository's S-Lab License 1.0 grants use and redistribution for non-commercial purposes subject to its conditions. These include retaining notices and restrictions on using contributor names for endorsement. It directs anyone needing commercial use or redistribution to contact the contributors. The reviewed project must therefore not be presented as an Apache 2.0 resource or as carrying a blanket commercial-use grant. Official S-Lab license.

Publicly readable code, a downloadable checkpoint and a paid hosted interface are different forms of access. Paying a provider does not itself establish the rights needed for your use of the underlying model, and a public demo is not evidence of an unrestricted license. For commercial work, establish the applicable permission and service terms before making the model part of a customer commitment. This page reports the project terms; it does not grant permission for a particular deployment.

The official downloader identifies a restoration checkpoint named codeformer.pth under the project's v0.1.0 release. That reference supports the existence of an official acquisition path; it is not a claim that this review downloaded, inspected or ran the binary. Keep the source revision, checkpoint origin and any locally recorded checksum with an evaluation record. A similarly named mirror should not silently replace the intended file. Official checkpoint downloader.

The README links author-maintained hosted options, but their capacity, prices, privacy practices and continuing operation were not tested here. For the family-photo exercise, choose a processing route that permits the intended handling of the photograph. Project availability does not resolve rights in the input image, permission to publish a person's portrait or the terms of optional dependencies. If any necessary permission is unclear, resolve it before submitting or redistributing material.

Run a bounded family-photo evaluation

Prepare a copy of a permitted scan and one less degraded portrait as a control. State the intended output, such as a private family display copy, and choose the features that must remain recognisable. Keep the original dimensions and record any cropping or resampling performed before inference. The control helps distinguish a problem specific to severe damage from a problem introduced by the chosen processing path; it is not a measured baseline until you run it.

Use the pinned official repository instructions to prepare an isolated environment, then record the versions actually installed and the device selected. The repository's installation recipe is not a compatibility certification for every current Python, PyTorch or operating-system combination. No installation was performed for this article. Resolve setup errors before comparing images, and keep the exact command and full log with each attempted run.

These are unexecuted comparison commands using flags present in the reviewed script. They assume the documented environment and required model files are already prepared. The paths are examples that you must replace with your permitted input and separate output locations. All three use the same ordinary-image workflow and leave the optional Real-ESRGAN background and face enhancement switches disabled. This does not disable the CLI's separate default --upscale 2 output scale.

python inference_codeformer.py -i inputs/family-photo.png -o results/family-w0 -w 0
python inference_codeformer.py -i inputs/family-photo.png -o results/family-w05 -w 0.5
python inference_codeformer.py -i inputs/family-photo.png -o results/family-w1 -w 1

Inspect face selection and alignment first, then compare each restored crop with the source, and finally inspect the recomposed image. Keep rejected outputs as well as a preferred derivative. If you add background or face upsampling, make it a separate recorded comparison after choosing an acceptable restoration. Otherwise, a visible improvement may come from another stage, and its contribution will be difficult to identify.

Accept a derivative with a face-specific worksheet

The following worksheet is an original proposed decision aid for this family-photo scenario. It contains no test results or quality ratings. Fill it with observations and saved artifacts before describing a run as successful.

Treat an attractive face that changes an important feature as a failed candidate for this task. Conversely, a conservative result may be useful even when some blur remains. Set the tolerance in terms of the intended photograph and recipient, not a generic sharpness target. The worksheet should make that choice explainable to someone who did not operate the model.

Set a correction budget before trying a larger collection. Count the time spent inspecting faces, rejecting outputs and repairing boundaries as part of the job. If a few photos require careful manual work, that may still be acceptable for a private album. If almost every face needs reconstruction, a batch workflow may remove less work than expected. Leave these observations blank until measured.

Diagnose the first artifact that is wrong

The reviewed script catches an exception during per-face restoration and substitutes the input crop for that face while reporting the failure. A saved image is therefore not sufficient evidence that model inference succeeded. Check the log and corresponding crop before counting an output as a restored face. This is especially relevant when a batch appears to finish but some faces are unchanged.

These are diagnostic hypotheses and inspection steps, not measured failure rates. Do not label every unchanged face an exception or every boundary problem an upsampler fault. Use the saved artifacts to identify the earliest relevant difference. That keeps a subsequent fix attached to an observed problem rather than to a general impression of image quality.

Measure resources for the complete job

The reviewed device helper can select MPS, CUDA or CPU under its implemented availability and version checks. That code path is not proof of successful execution on your machine. The optional Real-ESRGAN setup also warns about slow CPU processing. There is no tested minimum VRAM, universal per-face time or throughput promise in this evaluation. Official device-selection helper.

Measure initial downloads and startup separately from repeat inference. Record the full image dimensions, face count, selected detector, fidelity weight, optional enhancement stages and actual hardware. Include saving and manual inspection when estimating how long a usable photograph takes. A timing for one aligned face does not describe processing a large group photograph with background upsampling.

Video needs its own resource plan. In the reviewed script, the video path collects input frames and later output frames in memory. Do not describe it as a bounded streaming workflow or extrapolate a short clip to an unrestricted recording. The existence of a video input path also does not establish stable facial appearance across frames. If video is required, begin with a short permitted clip and inspect adjacent frames and memory use explicitly.

Choose the next restoration step from the evidence

For the family portrait, the immediate decision is whether an acceptable display derivative exists with tolerable manual correction. If it does, preserve the exact settings, original and edited file together. If the changes remain unacceptable, choose a clearer scan, careful manual retouching or a limited correction that leaves uncertain facial detail unresolved. A rejected model output is useful evidence for that choice.

A face-restoration model and a general upscaler address different parts of the photograph. Select another method according to the observed need, such as a scratch outside the face or a recomposition problem, rather than assuming a different model name will solve every defect. This review does not establish a CodeFormer-versus-GFPGAN winner or reproduce the paper's benchmarks.

Use the model directory to explore other model categories once the missing capability is clear. Before expanding to a collection or service, resolve the applicable permissions and establish the actual workload and acceptance results. Neither sharper detail, successful file creation nor a public repository alone completes that evaluation.