Skip to content

Add skimage.morphology.thin - #27

Merged
rapids-bot[bot] merged 7 commits into
rapidsai:branch-21.08from
grlee77:add_thin
Jun 28, 2021
Merged

rapids-bot[bot] merged 7 commits into
rapidsai:branch-21.08from
grlee77:add_thin

Conversation

@grlee77

@grlee77 grlee77 commented May 10, 2021

Copy link
Copy Markdown
Contributor

This PR adds one additional function for thinning binary images as shown here. This particular algorithm is 2D only. There is a second skeletonization algorithm in scikit-image with 3D support, but that one is currently Cython based and take more work to adapt.

@grlee77 grlee77 added feature request New feature or request non-breaking Introduces a non-breaking change labels May 10, 2021
@grlee77
grlee77 requested review from a team as code owners May 10, 2021 01:42

@ajschmidt8 ajschmidt8 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving ops-codeowner file changes

@quasiben

Copy link
Copy Markdown
Member

rerun tests

@jakirkham

Copy link
Copy Markdown
Member

From CI:

16:53:01 _____ ERROR collecting cucim/skimage/morphology/tests/test_skeletonize.py ______
16:53:01 ImportError while importing test module '/workspace/python/cucim/src/cucim/skimage/morphology/tests/test_skeletonize.py'.
16:53:01 Hint: make sure your test modules/packages have valid Python names.
16:53:01 Traceback:
16:53:01 /opt/conda/lib/python3.8/importlib/__init__.py:127: in import_module
16:53:01     return _bootstrap._gcd_import(name[level:], package, level)
16:53:01 src/cucim/skimage/morphology/tests/test_skeletonize.py:7: in <module>
16:53:01     from cucim.skimage.morphology import thin
16:53:01 E   ImportError: cannot import name 'thin' from 'cucim.skimage.morphology' (/opt/conda/lib/python3.8/site-packages/cucim/skimage/morphology/__init__.py)

Maybe there is still something to be worked out here?

@grlee77

grlee77 commented May 12, 2021

Copy link
Copy Markdown
Contributor Author

Maybe there is still something to be worked out here?

Yeah, this is odd. As far as I can tell the function is present in the __init__.py and I do not see the issue locally. It looks like a couple of other GPU-based jobs also succeeded on CI. Any ideas?

@jakirkham
jakirkham changed the base branch from branch-21.06 to branch-21.08 June 9, 2021 16:58
@jakirkham

Copy link
Copy Markdown
Member

Moving to 21.08 as 21.06 is being shipped. Hopefully we can get this in the new release

@grlee77

grlee77 commented Jun 9, 2021

Copy link
Copy Markdown
Contributor Author

I need to review the CI a bit more (maybe I am misunderstanding something), but why do I see in those logs in the sections on testing:

19:44:28     cucim-0.20.0a210511        |cuda_11.2_py38_g096cc79_6         1.1 MB  rapidsai-nightly

why is the cucim listed as above from rapidsai-nightly and not from some locally built version corresponding to this PR?

@jakirkham

Copy link
Copy Markdown
Member

Would wait for a bit before digging in. Suspect we are missing things like PR ( #53 )

@ajschmidt8

Copy link
Copy Markdown
Member

Looks like the 08 in the version number is getting interpreted as an octal value. rmm had a similar issue that was resolved in rapidsai/rmm#793

image

@gigony

gigony commented Jun 15, 2021

Copy link
Copy Markdown
Contributor

Looks like the 08 in the version number is getting interpreted as an octal value. rmm had a similar issue that was resolved in rapidsai/rmm#793

Thanks @ajschmidt8 for the pointer! I would look at this and fix this today.

rapids-bot Bot pushed a commit that referenced this pull request Jun 16, 2021
We need both version strings -- zero-padded(`08`) and normal integer `8` in c++ code.

```c++
#define CUCIM_VERSION_MAJOR 21
#define CUCIM_VERSION_MINOR 08
#define CUCIM_VERSION_PATCH 00
```

For zero-padded VERSION, it uses XSTR macro:
```c++
#define STR(x) #x
#define XSTR(x) STR(x)
```
For integer version (uint32_t), it uses  `std::atoi` with XSTR macro:
Example,
```c++
static_cast<uint32_t>(std::atoi(XSTR(CUCIM_VERSION_MINOR)))
```

Fix a build issue in #27.

Authors:
  - Gigon Bae (https://github.com/gigony)

Approvers:
  - https://github.com/jakirkham

URL: #59
@jakirkham

Copy link
Copy Markdown
Member

rerun tests

@jakirkham

Copy link
Copy Markdown
Member

CI is back to the error shown in comment ( #27 (comment) )

@grlee77

grlee77 commented Jun 16, 2021 •

Copy link
Copy Markdown
Contributor Author

Looking through the logs a bit more, cucim for this commit builds successfully with conda-build and ends up at:

/opt/conda/envs/rapids/conda-bld/linux-64/cucim-21.08.00a-cuda_11.2_py38_g4ca5c83_16.tar.bz2

The following command SHOULD be installing that version to use when running pytest:

gpuci_conda_retry install -y -c /workspace/ci/artifacts/cucim/cpu/.conda-bld -c /opt/conda/envs/rapids/conda-bld -c rapidsai-nightly 'rapids-build-env=21.08.*' libcucim cucim

However, there is an error about package inconsistency that is causing a different version of cucim to be retrieved from rapidsai-nightly instead! The conflicts can be seen in the log segment pasted below:

03:04:49 Collecting package metadata (current_repodata.json): ...working... done

03:06:51 Solving environment: ...working... 

03:06:51 The environment is inconsistent, please check the package plan carefully

03:06:51 The following packages are causing the inconsistency:

03:06:51 

03:06:51   - conda-forge/linux-64::nccl==2.9.9.1=hdc17891_0

03:06:51   - conda-forge/linux-64::cupy==9.1.0=py38ha69542f_0

03:06:51   - conda-forge/linux-64::libfaiss==1.7.0=cuda112h5bea7ad_8_cuda

03:06:51   - rapidsai-nightly/linux-64::ucx==1.9.0+gcd9efd3=cuda11.2_0

03:06:51   - rapidsai-nightly/linux-64::rapids-build-env==21.08.00a210615=cuda11.2_py38_g5cd3870_425

03:06:51 done

03:07:01 

03:07:01 ## Package Plan ##

03:07:01 

03:07:01   environment location: /opt/conda/envs/rapids

03:07:01 

03:07:01   added / updated specs:

03:07:01     - cucim

03:07:01     - libcucim

03:07:01     - rapids-build-env=21.08

03:07:01 

03:07:01 

03:07:01 The following packages will be downloaded:

03:07:01 

03:07:01     package                    |            build

03:07:01     ---------------------------|-----------------

03:07:01     cucim-0.20.0a210517        |cuda_11.2_py38_gbc3687c_13         1.1 MB  rapidsai-nightly

03:07:01     jbig-2.1                   |    h7f98852_2003          43 KB  conda-forge

03:07:01     libcucim-0.20.0a210517     |cuda11.2_gbc3687c_13         1.7 MB  rapidsai-nightly

03:07:01     ------------------------------------------------------------

03:07:01                                            Total:         2.8 MB

03:07:01 

03:07:01 The following NEW packages will be INSTALLED:

03:07:01 

03:07:01   cucim              rapidsai-nightly/linux-64::cucim-0.20.0a210517-cuda_11.2_py38_gbc3687c_13

03:07:01   jbig               conda-forge/linux-64::jbig-2.1-h7f98852_2003

03:07:01   libcucim           rapidsai-nightly/linux-64::libcucim-0.20.0a210517-cuda11.2_gbc3687c_13

03:07:01 

The above excerpt is from the following log:
https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci/job/cucim/job/prb/job/cucim-gpu-test/CUDA=11.2,GPU_LABEL=gpu,OS=ubuntu18.04,PYTHON=3.8/7/consoleFull

@grlee77

grlee77 commented Jun 16, 2021

Copy link
Copy Markdown
Contributor Author

Is there a simple way to constrain conda to only disallow cucim and libcucim to be installed from rapidsai-nightly and just error out if conflicts do not allow that for some reason? That would at least prevent falling back to unexpected versions from rapids-nightly and then trying to test against those.

@ajschmidt8

Copy link
Copy Markdown
Member

Is there a simple way to constrain conda to only disallow cucim and libcucim to be installed from rapidsai-nightly and just error out if conflicts do not allow that for some reason? That would at least prevent falling back to unexpected versions from rapids-nightly and then trying to test against those.

You could try splitting the following line and using --override-channels (docs link).

i.e.:

gpuci_conda_retry install  -y \
  -c /workspace/ci/artifacts/cucim/cpu/.conda-bld \
  -c /opt/conda/envs/rapids/conda-bld \
  -c rapidsai-nightly \
  'rapids-build-env=21.08.*' \
  libcucim \
  cucim

into

gpuci_conda_retry install -y \
  -c rapidsai-nightly \
  'rapids-build-env=21.08.*' 

gpuci_conda_retry install -y \
  --override-channels \
  -c /workspace/ci/artifacts/cucim/cpu/.conda-bld \
  -c /opt/conda/envs/rapids/conda-bld \
  libcucim \
  cucim

Though I'm not sure how/if this will work if cucim or libcucim have dependencies that exist in the rapidsai-nightly channel.

@grlee77

grlee77 commented Jun 17, 2021

Copy link
Copy Markdown
Contributor Author

Splitting the command forced the local version to be used so that it now errors out with more messages about the conflicts. The reported conflicts look compiler/C library related (__glibc, libgcc-ng and _openmp_mutex). Do we still have some inconsistency between GCC versions used here vs. other RAPIDS projects (or for CuPy on conda-forge)?

The following claiming there is a CUDA mismatch looks odd to me as well (it appears both the detected and expected are actually 11.2?):

20:24:34 cucim -> libcucim=21.08.00aThe following specifications were found to be incompatible with your CUDA driver:

20:24:34 

20:24:34   - feature:/linux-64::__cuda==11.2=0

20:24:34   - feature:|@/linux-64::__cuda==11.2=0

20:24:34 

20:24:34 Your installed CUDA driver is: 11.2

Details
20:21:48 ++ gpuci_conda_retry install -y -c /workspace/ci/artifacts/cucim/cpu/.conda-bld -c /opt/conda/envs/rapids/conda-bld --override-channels libcucim cucim

20:22:02 Collecting package metadata (current_repodata.json): ...working... done

20:22:51 Solving environment: ...working... 

20:22:51 The environment is inconsistent, please check the package plan carefully

20:22:51 The following packages are causing the inconsistency:

20:22:51 

20:22:51   - conda-forge/linux-64::nccl==2.9.9.1=hdc17891_0

20:22:51   - conda-forge/linux-64::cupy==9.1.0=py38ha69542f_0

20:22:51   - conda-forge/linux-64::libfaiss==1.7.0=cuda112h5bea7ad_8_cuda

20:22:51   - rapidsai-nightly/linux-64::ucx==1.9.0+gcd9efd3=cuda11.2_0

20:22:51   - rapidsai-nightly/linux-64::rapids-build-env==21.08.00a210616=cuda11.2_py38_g5cd3870_425

20:22:51 failed with initial frozen solve. Retrying with flexible solve.

20:23:13 Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.

20:23:23 Collecting package metadata (repodata.json): ...working... done

20:24:12 Solving environment: ...working... 

20:24:12 The environment is inconsistent, please check the package plan carefully

20:24:12 The following packages are causing the inconsistency:

20:24:12 

20:24:12   - conda-forge/linux-64::nccl==2.9.9.1=hdc17891_0

20:24:12   - conda-forge/linux-64::cupy==9.1.0=py38ha69542f_0

20:24:12   - conda-forge/linux-64::libfaiss==1.7.0=cuda112h5bea7ad_8_cuda

20:24:12   - rapidsai-nightly/linux-64::ucx==1.9.0+gcd9efd3=cuda11.2_0

20:24:12   - rapidsai-nightly/linux-64::rapids-build-env==21.08.00a210616=cuda11.2_py38_g5cd3870_425

20:24:12 failed with initial frozen solve. Retrying with flexible solve.

20:24:34 Solving environment: ...working... 
Building graph of deps:   0%|          | 0/5 [00:00<?, ?it/s]
Examining libcucim:   0%|          | 0/5 [00:00<?, ?it/s]    
Examining python=3.8:  20%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½        | 1/5 [00:00<00:00, 402.99it/s]
Examining cucim:  40%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½      | 2/5 [00:00<00:00, 359.86it/s]     
Examining @/linux-64::__cuda==11.2=0:  60%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½    | 3/5 [00:00<00:00, 252.10it/s]
Examining @/linux-64::__glibc==2.27=0:  80%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½  | 4/5 [00:00<00:00, 328.40it/s]
                                                                                     

Determining conflicts:   0%|          | 0/5 [00:00<?, ?it/s]
Examining conflict for libcucim cucim:   0%|          | 0/5 [00:00<?, ?it/s]
Examining conflict for libcucim python cucim:  20%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½        | 1/5 [00:00<00:00, 1041.80it/s]
Examining conflict for libcucim __glibc cucim:  40%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½      | 2/5 [00:00<00:00, 46.24it/s] 
Examining conflict for python cucim:  60%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½    | 3/5 [00:00<00:00, 44.97it/s]          
Examining conflict for __cuda:  80%|Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½Γ’οΏ½οΏ½  | 4/5 [00:00<00:00, 45.46it/s]      
                                                                            

20:24:34 Found conflicts! Looking for incompatible packages.

20:24:34 This can take several minutes.  Press CTRL-C to abort.

20:24:34 failed

20:24:34 

20:24:34 UnsatisfiableError: The following specifications were found to be incompatible with each other:

20:24:34 

20:24:34 Output in format: Requested package -> Available versions

20:24:34 

20:24:34 Package __glibc conflicts for:

20:24:34 cucim -> cupy=9 -> __glibc[version='>=2.17']

20:24:34 cucim -> __glibc[version='>=2.17,<3.0.a0']

20:24:34 @|@/linux-64::__glibc==2.27=0

20:24:34 @/linux-64::__glibc==2.27=0

20:24:34 

20:24:34 Package libgcc-ng conflicts for:

20:24:34 libcucim -> libgcc-ng[version='>=9.3.0']

20:24:34 libcucim -> jpeg[version='>=9d,<10a'] -> libgcc-ng[version='>=7.5.0']

20:24:34 

20:24:34 Package _openmp_mutex conflicts for:

20:24:34 python=3.8 -> libgcc-ng[version='>=9.3.0'] -> _openmp_mutex[version='>=4.5']

20:24:34 libcucim -> libgcc-ng[version='>=9.3.0'] -> _openmp_mutex[version='>=4.5']

20:24:34 cucim -> libgcc-ng[version='>=9.3.0'] -> _openmp_mutex[version='>=4.5']

20:24:34 

20:24:34 Package libcucim conflicts for:

20:24:34 libcucim

20:24:34 cucim -> libcucim=21.08.00aThe following specifications were found to be incompatible with your CUDA driver:

20:24:34 

20:24:34   - feature:/linux-64::__cuda==11.2=0

20:24:34   - feature:|@/linux-64::__cuda==11.2=0

20:24:34 

20:24:34 Your installed CUDA driver is: 11.2

20:24:34 

20:24:34 

20:24:34     [gpuci_conda_retry] conda returned exit code: 1

20:24:34     [gpuci_conda_retry] Exiting, no retryable conda errors detected: 'ChecksumMismatchError:' or 'CondaHTTPError:' or 'JSONDecodeError:' or 'ChunkedEncodingError:' or 'CondaMultiError:'

20:24:34     [gpuci_conda_retry] 

20:24:34 Build step 'Execute shell' marked build as failure

20:24:37 Recording test results

20:24:38 [Checks API] No suitable checks publisher found.

20:24:38 [Set GitHub commit status (universal)] ERROR on repos [GHRepository@61878e63[nodeId=MDEwOlJlcG9zaXRvcnkzNTYzNTYzMzY=,description=

@ajschmidt8

Copy link
Copy Markdown
Member

Splitting the command forced the local version to be used so that it now errors out with more messages about the conflicts. The reported conflicts look compiler/C library related (__glibc, libgcc-ng and _openmp_mutex). Do we still have some inconsistency between GCC versions used here vs. other RAPIDS projects (or for CuPy on conda-forge)?

The following claiming there is a CUDA mismatch looks odd to me as well (it appears both the detected and expected are actually 11.2?):

I'm not great with compiler stuff, but I'm wondering if @jakirkham has any thoughts or input on this. I checked the package versions from the integration repo to see if they match up with both the libcuim and cucim conda recipes, and it looks like they do.

@gigony

gigony commented Jun 18, 2021

Copy link
Copy Markdown
Contributor

Splitting the command forced the local version to be used so that it now errors out with more messages about the conflicts. The reported conflicts look compiler/C library related (__glibc, libgcc-ng and _openmp_mutex). Do we still have some inconsistency between GCC versions used here vs. other RAPIDS projects (or for CuPy on conda-forge)?

The following claiming there is a CUDA mismatch looks odd to me as well (it appears both the detected and expected are actually 11.2?):

I'm not great with compiler stuff, but I'm wondering if @jakirkham has any thoughts or input on this. I checked the package versions from the integration repo to see if they match up with both the libcuim and cucim conda recipes, and it looks like they do.

#62 has the same issue with this.

@ajschmidt8
In my local machine, just creating a new environment (in no existing conda environment) with rapids-build-env is failing.

conda create -n rapids-env -c rapidsai-nightly rapids-build-env

Details
gbae@GIGONY-DESKTOP:~/repo/cucim$ conda create -n rapids-env -c rapidsai-nightly rapids-build-env
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: | 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                                                        -  

UnsatisfiableError: The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.31=0
  - rapids-build-env -> ucx -> __glibc[version='>=2.17|>=2.17,<3.0.a0']

Your installed version is: 2.31



gbae@GIGONY-DESKTOP:~/repo/cucim$ conda info

     active environment : None
            shell level : 0
       user config file : /home/gbae/.condarc
 populated config files : /home/gbae/.condarc
          conda version : 4.10.1
    conda-build version : 3.21.4
         python version : 3.8.5.final.0
       virtual packages : __linux=5.10.16.3=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/gbae/miniconda3  (writable)
      conda av data dir : /home/gbae/miniconda3/etc/conda
  conda av metadata url : https://repo.anaconda.com/pkgs/main
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/gbae/miniconda3/pkgs
                          /home/gbae/.conda/pkgs
       envs directories : /home/gbae/miniconda3/envs
                          /home/gbae/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.1 requests/2.25.1 CPython/3.8.5 Linux/5.10.16.3-microsoft-standard-WSL2 ubuntu/20.04.1 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

@jakirkham

Copy link
Copy Markdown
Member

Can you please try specifying cudatoolkit=<some CUDA version> in that command @gigony?

@gigony

gigony commented Jun 18, 2021

Copy link
Copy Markdown
Contributor

Can you please try specifying cudatoolkit=<some CUDA version> in that command @gigony?

It doesn't help :(

gbae@GIGONY-DESKTOP:~/repo/cucim$ conda create -n rapids-env -c rapidsai-nightly rapids-build-env cudatoolkit=11.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: - 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
Examining rapids-build-env:  50%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ                                                                        | 3/6 [00:00<00:00, 5790.57it- ^failed                                                                                                                                                                                                           

CondaError: KeyboardInterrupt

Even this command is hanging

gbae@GIGONY-DESKTOP:~/repo/cucim$ conda create -n rapids-env  -c gpuci -c rapidsai-nightly -c rapidsai -c nvidia -c pytorch -c conda-forge rapids-build-env cudatoolkit=11.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: \ 

Comment thread ci/gpu/build.sh Resolved
ajschmidt8 and others added 2 commits June 28, 2021 15:40
We need both version strings -- zero-padded(`08`) and normal integer `8` in c++ code.

```c++
#define CUCIM_VERSION_MAJOR 21
#define CUCIM_VERSION_MINOR 08
#define CUCIM_VERSION_PATCH 00
```

For zero-padded VERSION, it uses XSTR macro:
```c++
#define STR(x) #x
#define XSTR(x) STR(x)
```
For integer version (uint32_t), it uses  `std::atoi` with XSTR macro:
Example,
```c++
static_cast<uint32_t>(std::atoi(XSTR(CUCIM_VERSION_MINOR)))
```

Fix a build issue in rapidsai#27.

Authors:
  - Gigon Bae (https://github.com/gigony)

Approvers:
  - https://github.com/jakirkham

URL: rapidsai#59
@grlee77
grlee77 requested a review from a team as a code owner June 28, 2021 19:41
@grlee77

grlee77 commented Jun 28, 2021

Copy link
Copy Markdown
Contributor Author

Okay, the conda conflicts seem to have been resolved elsewhere and this is working now!

@jakirkham

Copy link
Copy Markdown
Member

Yeah I suspect we were just running into the same Arrow issues we saw elsewhere in RAPIDS. Now that those have been resolved this has cleared up

@jakirkham

Copy link
Copy Markdown
Member

@gpucibot merge

@rapids-bot
rapids-bot Bot merged commit 8da6ca9 into rapidsai:branch-21.08 Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants