Skip to content

Fetch request body as application/x-www-form-urlencodedΒ #633

Closed

Description

@duchoang

While upgrading ScalaJS from 1.x to 2.x, we are hitting to this issue with sending data with urlencoded type (in version 1.x we were using Ajax, now we must switch to fetch())

Example with this simple code:

import org.scalajs.dom._

val headers = new Headers()
headers.set("content-type", "application/x-www-form-urlencoded")
Fetch
      .fetch(
        "/test-api",
        new RequestInit {
          method = HttpMethod.POST
          body = "key1=value1&key2=value2"
          headers = headers
        }
      )

When executing this script in browser, the content-type header of fetch request is changed to be text/plain;charset=UTF-8. It seems ScalaJs itselves detects the body as string and decide to change the content-type to text/plain which is wrong.

If I use the same code above directly with javascript here (testing by openning the chrome console), the fetch request is sent correctly with content-type urlencoded.

window.fetch('/test-api', {
  method: 'POST',
  headers: {
  	'content-type': 'application/x-www-form-urlencoded'
  },
  body: 'key1=value1&key2=value2'
})

Could you please show me how to use this properly?

While I search some solution for this, I notice one workaround for this is to use this type URLSearchParams in the body of fetch request, this will set the content-type automatically to urlencoded. But sadly the type of request body RequestInit doesnt support this yet:

Blob | BufferSource | FormData | String // todo: add URLSearchParams

Activity

  1. armanbilge commented on Nov 16, 2021

    @armanbilge
    Member

    Thanks for reporting. That is indeed strange, I can't understand why scala-js-dom would do something different than vanilla JavaScript: it is purely a facade for the JavaScript API.

    What browser are you testing with?

    Can you try setting the headers in Scala.js like this:

    Fetch
          .fetch(
            "/test-api",
            new RequestInit {
              method = HttpMethod.POST
              body = "key1=value1&key2=value2"
              headers = js.Dictionary("content-type" -> "application/x-www-form-urlencoded")
            }
          )

    This matches your JavaScript code more closely.

    Similarly, can you try changing the JavaScript code to:

    var headers = new Headers();
    headers.set("content-type", "application/x-www-form-urlencoded");
    window.fetch('/test-api', {
      method: 'POST',
      headers: headers,
      body: 'key1=value1&key2=value2'
    })

    This matches the original Scala.js code that has the problem.

    Please try those and let me know what happens.


    Regarding URLSearchParams...

    But sadly the type of request body RequestInit doesnt support this yet:

    It seems we have a facade for URLSearchParams already, it just needs to be added to the union there. Would you like to make a PR? :)

    class URLSearchParams extends js.Iterable[js.Tuple2[String, String]] {

  2. duchoang commented on Nov 16, 2021

    @duchoang
    Author

    Thanks @armanbilge for your detail reply, to make it easier testing, I setup a new project to demonstrate this issue here:

    https://github.com/duchoang/scalajs-fetch-urlencoded/blob/main/client/src/main/scala/example/client.scala#L11

    Button1 is using Headers type and button2 is using js.Dictionary type.

    To run this example, you can run these sbt command:

    $> sbt
    sbt:scalajs-fetch-urlencoded> compile
    sbt:scalajs-fetch-urlencoded> webserver/run

    Open example web at http://localhost:8080/ and click the first button, the fetch request is sent with header content-type: text/plain

    image

    while for second button, it is sent correctly with form-urlencoded

    image

    Thanks for the suggestion with js.Dictionary, it works for our case, but why it is failing with Headers type is still a mystery, I leave this to your side to investigate more, feel free to check out the above example code and test.

    In addition, I'm happy to raise a PR to let BodyInit type support this URLSearchParams, will do it soon.

  3. armanbilge commented on Nov 16, 2021

    @armanbilge
    Member

    Thank you for setting up the example! Very helpful. Look forward to the PR :)

    Glad you have a working solution for now, we'll investigate the problem with Headers.

  4. added this to the v2.1.0 milestone on Nov 16, 2021
  5. armanbilge commented on Nov 16, 2021

    @armanbilge
    Member

    Ah, this is quite silly actually! This code will work.

    val myHeaders = new Headers()
    myHeaders.set("content-type", "application/x-www-form-urlencoded")
    Fetch
          .fetch(
            "/test-api",
            new RequestInit {
              method = HttpMethod.POST
              body = "key1=value1&key2=value2"
              headers = myHeaders
            }
          )

    headers was cyclically referencing itself, not the variable defined above.

  6. removed this from the v2.1.0 milestone on Nov 16, 2021
  7. yurique commented on Nov 16, 2021

    @yurique

    Oh wow, I would never have guessed! :)

  8. yurique commented on Nov 16, 2021

    @yurique

    BTW, I think I did a similar thing for a different trait, but one of the scalac warnings got triggered (something like "this thing is doing nothing except referencing itself", might have been scala3)

  9. armanbilge commented on Nov 16, 2021

    @armanbilge
    Member

    Yes, I think also enabling warnings about unused variables (which Scala 2 has) would help catch thisβ€”not the cyclic reference of course, but the unused Headers above :)

    -Xlint:unused
        Enable -Wunused:imports,privates,locals,implicits.
    
  10. duchoang commented on Nov 17, 2021

    @duchoang
    Author

    @armanbilge wow thanks for your debugging, didn't notice that self-referencing πŸ˜‚

  11. armanbilge commented on Nov 17, 2021

    @armanbilge
    Member

    @duchoang btw I noticed you are using http4s in your example project, you may find this project useful: https://github.com/http4s/http4s-dom

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions