Repository navigation
[BUG] Compression scheme 33003 tile decoding is not implemented. #17
Description
Describe the bug
For Aperio svs files, I run into a cryptic error:
/slide.svs: Compression scheme 33003 tile decoding is not implemented.
However, cuCMI still loads a tile and doesn't raise but the color are wrong or it is entirely black. See the following grid of tiles:
Are all OpenSlide vendors supported? Is there a list of all supported vendors somewhere? I could not find it.
Steps/Code to reproduce bug
# Download the TCGA-F4-6459-01Z-00-DX1 slide from https://portal.gdc.cancer.gov/cases/9fd08502-355b-4f5a-a25c-73ec7184f6d3?bioId=57f4136d-d6aa-4537-ba73-f21fe0374005
import numpy as np
from PIL import Image
from cucmi import CuImage
slide = CuImage("TCGA-F4-6459-01Z-00-DX1")
tile = slide.read_region((0, 0), (224, 224), 0)
Image.fromarray(np.asarray(tile))
Activity
gigony commentedon Apr 22, 2021
Hello @OlivierDehaene !
Thank you for your interest in cuCIM!
Compression scheme 33003 is actually JPEG2000 which is used in Aperio SVS format.
cuCIM currently supports tiled RGB TIFF image(Generic TIFF or Philips TIFF) with JPEG or Deflate compressed and doesn't support SVS format yet.
However, we have a plan to support it in the near future.
The tentative plan is specified in the following link (I would move the content to Github wiki page later):
https://github.com/rapidsai/cucim/blob/branch-0.20/python/cucim/docs/roadmap/index.md#facalendar-alttext-info-mr-1-v020
If cuCIM encounters TIFF (SVS is a variant of TIFF format) file that is not JPEG/Deflate compressed RGB format, it just delegates its process to libtiff's TIFFRGBAImageGet() method and it seems that warning/error are not properly handled.
Let me fix the issue by raising an error until cuCIM supports SVS format properly.
(I am currently working on providing a cache mechanism to cuCIM so that the performance could be greatly improved)
Thank you for the report! We are also eager to support various image formats so stay tuned!
Check compression method used in the image (rapidsai#17)
Check compression method used in the image (rapidsai#17)
OlivierDehaene commentedon Apr 23, 2021
Thank you for your answer @gigony!
Amazing! I have been looking for a better alternative to OpenSlide for a while now and if cuCIM starts to support Aperio that would be great :)
Will we be able to disable the cache when doing random tile reads? We generally disable the OpenSlide one when training.
gigony commentedon Apr 23, 2021
@OlivierDehaene
I thought cuCIM can allocate some memory for cache by default and the user can increase the cache size programmatically. I assumed it wouldn't be a problem if we use a cache memory per 'parent process' and cache memory is shared among subprocesses (whereas OpenSlide seems to allocate cache memory per file).
However, you're right. It is better to have an option to disable cache at all as cache wouldn't much help when reading random partial images from a huge image pool during training.
Now I am thinking of allocating cache memory only when requested :)
Thank you so much for your valuable feedback!
Check compression method used in the image (#17) (#19)
jakirkham commentedon May 5, 2021
Some packages should be pushed to
rapidsai-nightlysoon with Gigon's fix. Also this should show up in the next release. Please let us know if you run into any more issues 🙂Support raw RGB tiled TIFF (#108)
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder
Support Aperio SVS with CPU LZW and jpeg2k decoder (#141)