Skip to content

fix 3.0.0-SNAPSHOT build issuesย #431

Closed

Description

@pjfanning

Compilation fails due to major changes in jackson-databind

https://travis-ci.org/FasterXML/jackson-module-scala/jobs/588055923

Activity

  1. cowtowncoder commented on Sep 22, 2019

    @cowtowncoder
    Member

    Yeah, this will require quite a bit of work, likely. Let me know if I can help: Kotlin module has some challenges too, but I did help with some changes there.

  2. added
    3.xSomething that likely has to be done in 3.x, not 2.x
    on Sep 23, 2019
  3. cowtowncoder commented on Jul 25, 2020

    @cowtowncoder
    Member

    @pjfanning One question/suggestion, more related to 2.12 (but eventually to 3.0/master too): I created this project:

    https://github.com/FasterXML/jackson-integration-tests

    to allow basic sanity/smoke tests; something that would run simple tests to ensure that a set (like 2.12.0-SNAPSHOT, 3.0.0-SNAPSHOT) of components in their latest publish snapshots versions still works. I have added tests for dataformats, some of datatypes (joda, guava), as well as certain combinations (Joda + XML/CSV/Properties wrt timestamp values).

    I am not sure exactly how, but I wonder if there was a way to invoke some Scala mapper sanity tests from Java, built by Maven, to do simplest of validations that a scala-supporting object mapper can be constructed and used on something that exercises its functionality?
    I suspect the main challenge is that of building scala types for testing; perhaps integration-tests needs to be split to either have a support package(s) of value types (Kotlin no doubt needs something similar), or even just fully separate sub-projects (making current main-level project a sub-project).

    At any rate, I would really like to quickly find out if changes I am making break Scala module: I just integrated PR for

    FasterXML/jackson-databind#1296

    and doing refactoring for it. I will try to keep things backwards compatible for 2.12 obviously, but sometimes there are dependencies I do not see (Scala and Kotlin typically, although of course there are many 3rd party extensions beyond this). And with 3.x I will simply remove deprecated methods so there is more work anyway.

  4. pjfanning commented on Jul 25, 2020

    @pjfanning
    MemberAuthor

    I'm not sure - all that can be done is to copy lots of the code and tests from jackson-module-scala into jackson-integration-tests - I'd be reluctant to maintain 2 copies of the code.

    I have nightly builds running jackson-module-scala on travis but recently my travis account has had a problem that doesn't let me set up new builds for github orgs that I am member of. I need to either follow up with travis or try something else like github actions.

    I'm afraid master version of jackson-module-scala was well broken before I even had a chance to set up builds. I have it compiling and tests running but many are broken - many tests are probably broken for similar reasons.

  5. pjfanning commented on Jul 25, 2020

    @pjfanning
    MemberAuthor

    @cowtowncoder a lot of the broken tests appear to be because the DeserializerResolvers in master do not appear to have calls made to findReferenceDeserializer - only to findBeanDeserializer.

    Tests that pass in v2.12 and that fail in master have lots of calls to findReferenceDeserializer in 2.12 but none in master.

    https://github.com/FasterXML/jackson-module-scala/blob/master/src/main/scala/com/fasterxml/jackson/module/scala/deser/EitherDeserializer.scala#L115

  6. cowtowncoder commented on Jul 25, 2020

    @cowtowncoder
    Member

    @pjfanning Quick clarification wrt jackson-integration-tests: the idea would be a very small set of tests, similar to "hello world" style, even just one read, one write or such. To catch catastrophic cases. I wouldn't want to maintain separate sets of tests either.
    One other benefit is ability to test combinations (like Scala + CSV, Scala + XML), without adding dependencies across components. Those tests would not be duplicated FWTW.
    But it is up to you of course, I will mention this as something I plan to make use of and wanted to mention.

  7. cowtowncoder commented on Jul 25, 2020

    @cowtowncoder
    Member

    @pjfanning On findReferenceDeserializer: that sounds odd, as if anything, 3.0 should be making more accurate calls -- although without some of fallbacks, which may be the reason for breakage.

    I would guess 2 likely sources of issues:

    1. Some calls that relied on fallback (first check reference deserializer, if not found, try bean deserializer?) do not implement fallbacks any more
    2. Type refinement is somehow not working so calls are being made to wrong lookup methods

    Would it be possible to test to see if calls expected to go to findReferenceDeserializer() end up on findBeanDeserializer() or vice versa? Databind itself has only one reference type (for AtomicReference) and while that is tested is not extensively tested.

  8. pjfanning commented on Jul 25, 2020

    @pjfanning
    MemberAuthor

    I'm not sure when I'll next have chance to spend time on this.

    With reference types, wouldn't java.util.Optional be a reference type (akin to scala.Option and the master tests are failing for scala.Option)?

  9. cowtowncoder commented on Jul 25, 2020

    @cowtowncoder
    Member

    Yes, java.util.Optional is a reference type for sure; refined as such by jackson-module-java8 (in 2.x), and jackson-databind itself in master.
    And I think scala.Option should become one: but it is up to module to make sure that happens with TypeModifier implementation. But I think this is done by Scala module, both for 2.x and 3.0, and should also be tested that this occurs (by resolving via TypeFactory on which appropriate TypeModifier has been registered).

  10. pjfanning commented on Jul 26, 2020

    @pjfanning
    MemberAuthor

    In 2.12, the reference type deserializer is created in this stack - this does not happen in master, despite the same ReferenceType being in scope.

    java.lang.Exception: >>>>>> EitherDeserializerResolver findReferenceDeserializer class scala.util.Either
    	at com.fasterxml.jackson.module.scala.deser.EitherDeserializerResolver$.findReferenceDeserializer(EitherDeserializer.scala:121)
    	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCustomReferenceDeserializer(BasicDeserializerFactory.java:1976)
    	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.createReferenceDeserializer(BasicDeserializerFactory.java:1553)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    	at com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:581)
    	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:524)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:591)
    	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4684)
    	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4545)
    	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3497)
    	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3480)
    
  11. pjfanning commented on Jul 26, 2020

    @pjfanning
    MemberAuthor

    I found what was breaking the scala Either tests on master but quite a lot more stuff to fix still.

    @cowtowncoder could you look at FasterXML/jackson-module-jsonSchema#140 ?

  12. cowtowncoder commented on Jul 26, 2020

    @cowtowncoder
    Member

    @pjfanning Ok good wrt Either tests. Added a note on json-schema module: at this point I have no plans to support it for 3.x, will be end-of-life'd (not much work on 2.x either).

  13. 52 remaining items

  14. cowtowncoder commented on Feb 11, 2021

    @cowtowncoder
    Member

    Ok that makes sense. The problem seems to be that introspection does not find creator property for some reason. Not MetricPath type itself. In fact it would seem that it could as well be any other type (String) and same should occur?

    At this point, would it make sense to create a new issue for this one problem. Not sure if there are other ones, but discussion here has gotten bit long and covers a few different issues, most resolved?

  15. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    I found some code in the Scala annotation classes that is not behaving as it does in Jackson 2 - let me investigate this - it could be the cause of the issue here

  16. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    @cowtowncoder a possible/probable cause is that the jackson-module-scala was modified in jackson 3 because the interface in jackson-databind changed.

    ValueInstantiators findValueInstantiator(DeserializationConfig var1, BeanDescription var2, ValueInstantiator var3) has dropped the final param - the defaultValueInstantiator

    The scala code now struggles to replace that default. Any suggestions as to how to get a default instantiator in jackson 3?

  17. cowtowncoder commented on Feb 11, 2021

    @cowtowncoder
    Member

    @pjfanning Let me see, I think this was just split in two parts or so.

    Ok, yes; so, there is new modifyValueInstantiator() which is what you want to use in case you want to use the default one: so existing findValueInstantiator() is meant for full override case; and then modifyValueInstantiator() for things like delegation.

  18. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    I have no value instantiator so can't modify it. Is there any way to create a new default instantiator that has all the creator properties calculated - that's what I get in jackson 2 - automatically but now I am left with no instantiator and need to create from scratch .

    findValueInstantiator(config: DeserializationConfig, ...) -- because this method does not give access to a full DeserializationContext, it is hard to try BeanDeserializerBuilder to try to get a value instantiator from it

  19. cowtowncoder commented on Feb 11, 2021

    @cowtowncoder
    Member

    Please look at signature:

        default ValueInstantiator modifyValueInstantiator(DeserializationConfig config,
                BeanDescription beanDesc, ValueInstantiator defaultInstantiator);
    

    you are given the default instantiator UNLESS earlier call to

        public ValueInstantiator findValueInstantiator(DeserializationConfig config,
                BeanDescription beanDesc);
    

    already returned a ValueInstantiator.

    So basically earlier single call was split in two parts: one to ask for instantiator, and if none given, a default POJO instantiator is created and passed to give another chance for custom variants.

    So if you want to use that default instantiator it is given and you may return it as-is, or create different one based on it (or just ignore).

  20. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor
        default ValueInstantiator modifyValueInstantiator(DeserializationConfig config,
                BeanDescription beanDesc, ValueInstantiator defaultInstantiator) {
            return defaultInstantiator;
        }
    

    I have no defaultInstantiator to pass in

  21. cowtowncoder commented on Feb 11, 2021

    @cowtowncoder
    Member

    Ok now I am confused. This is a callback method of ValueInstantiators interface.
    Scala module should not usually be calling it.... it is called by Jackson databind (specifically BasicDeserializerFactory). Modules would implement ValueInstantiators callback to be able to provide alternative instantiator. That handling was split from 1 to 2 methods.

    If Scala module does need to call it, where is the code? Maybe I can help with it.

  22. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    In jackson 2, the perfect value instantiator is created in BasicDeserializer.findValueInstantiator -- and specifically when this is called instantiator = this._constructDefaultValueInstantiator(ctxt, beanDesc);

    This instantiator is then used when the call to the old ValueInstantiators findValueInstantiator(DeserializationConfig var1, BeanDescription var2, ValueInstantiator var3) function is called

  23. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    I think I fixed it - I made the findValueInstantiator return null so now the modifyValueInstantiator is called with the instantiator created in the BasicDeserializer

  24. pjfanning commented on Feb 11, 2021

    @pjfanning
    MemberAuthor

    We're now down to 2 or 3 commented out tests in master that pass in jackson v2 and that fail in v3 - today's fix fixed about 5 broken tests

  25. cowtowncoder commented on Feb 11, 2021

    @cowtowncoder
    Member

    @pjfanning Ok good, yes, findValueInstantiator() should return null to indicate "go ahead construct default one" and then that one will be passed for possible modification/replacement. So fix makes sense.

    Getting very close to parity which is good!

  26. pjfanning commented on Jul 31, 2021

    @pjfanning
    MemberAuthor

    created #536 for follow on work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.xSomething that likely has to be done in 3.x, not 2.x

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions