Skip to content

[BUG] Hue color jitter on an all gray or all white image returns a black image #490

Closed

Description

@benlansdell

Describe the bug

Hue jitter, via cucim.core.operations.color.color_jitter, on images with a lot (or all) gray seems to overflow and give either black pixels, or an all-black image.

I noticed this when using augmenting images of planes in, e.g., CIFAR: color_jitter with a non-zero hue parameter would render some sky pixels black. I found this issue only comes up when the hue parameter is non-zero -- other jitter parameters didn't seem to affect the behavior. Further experimenting showed that inputting an image of all white or all one shade of gray resulted in color_jitter outputting an image of all zeros.

More specifically:

import cupy as cp
import cucim.core.operations.color as ccl

x_cupy = cp.ones((3, 64, 64)).astype(cp.uint8)*255
x_jitter = ccl.color_jitter(x_cupy,0,0,0,0.1)

#Outputs all zeros, i.e.
(x_jitter == 0).all() # is True

The same behavior occurs for any gray image, e.g. if I make x_cupy a tensor of all intensity 128.

Perhaps this is some rounding/overflow issue since color_jitter seems to convert all inputs to uint8?

Expected behavior

I would expect the output to not be all black but, if the image is all gray then hue jitter should have no effect -- it should output the same gray image. The torch implementation, for instance, behaves how I would expect:

import torch
import numpy as np 
import torchvision
x_np = np.ones((3, 64, 64)).astype(np.uint8)*255
x_torch = torch.tensor(x_np)
x_torch_jitter = torchvision.transforms.ColorJitter(0, 0, 0, 0.1)(x_torch)

#Jittered output is same as input image
(x_torch_jitter == 255).all() # is True

Environment

Running Ubuntu 20.04. cudatoolkit 11.6.2.

Installed cuCIM with pip.
python==3.8.13
cupy==11.2.0
cucim==22.10.00

(torch version for the above example == 1.13.0)

Other notes

  • I came across this issue when using MONAI's RandCuCIM transform for data augmentation, as it calls cucim's color_jitter.
  • I've tried a number of hue parameters -- anything non-zero seems to create the issue. Though the docstring seems to suggest anything between [-0.5, 0.5] is accepted, including negative values, negative values in fact return ValueError: If hue is a single number, it must be non negative. But I suppose that's a separate issue.

Activity

  1. benlansdell commented on Jan 29, 2023

    @benlansdell
    ContributorAuthor

    Looking at the CUDA kernel, here, it seems like the issue lies on line 208, here.

    If s == 0 it sets the whole output to zero. But I think HSV is only black if v == 0.

    Line 208 should be changed to:

        if (v == 0) {
    

    ??

  2. added a commit that references this issue on Jan 30, 2023
    3d29e7d
  3. added a commit that references this issue on Mar 27, 2024
    77581d1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions