Skip to content

Typescript translation of union-type arrays is incorrectΒ #160

Closed

Description

@nielstron

At least in the huggingface version it seems that the translation of "arrays of unions" is consistently incorrect. The error seems quite straightforward: When writing the type of an array of union (i.e. number | string), one needs to put parentheses around the union for the type to be correctly interpreted.

For example

number | string[] // this could be either a number or an array of strings
(number | string)[] // this is an array containing possibly both numbers and strings

Here is an example where due to this issue already the unit tests don't type correctly:

Instance 405

001 function check_tuplex(tuplex: string| number[], tuple1: any): boolean {
...
007 
008 declare var require: any;
009 const assert = require('node:assert');
010 
011 
012 function test() {
013   let candidate = check_tuplex;
014   assert.deepEqual(candidate(["w", 3, "r", "e", "s", "o", "u", "r", "c", "e"], "r"),true);
015   assert.deepEqual(candidate(["w", 3, "r", "e", "s", "o", "u", "r", "c", "e"], "5"),false);
016   assert.deepEqual(candidate(["w", 3, "r", "e", "s", "o", "u", "r", "c", "e"], 3),true);
017 }

The compiler error message points out that the parameters passed in the test are inconsistent with the annotation - because the parentheses are missing.

../../../../../tmp/tmppk_u13br.ts(14,30): error TS2345: Argument of type '(string | number)[]' is not assignable to parameter of type 'string | number[]'.

Other instances:

  • 253
  • 278
  • 294
  • 297

Activity

  1. arjunguha commented on Nov 3, 2024

    @arjunguha
    Member

    Thanks, looks like a bug. The report mentions problems for MBPP. Do you know off-hand if this affects HumanEval?

  2. nielstron commented on Nov 3, 2024

    @nielstron
    Author

    I didn't see any cases affecting humaneval, but it might be better to double check

  3. added a commit that references this issue on Nov 13, 2024
  4. arjunguha commented on Nov 13, 2024

    @arjunguha
    Member

    Thanks for reporting this. I think I have a fix. Here are the changes to the prompts:

    === humaneval-ts_prompt_HumanEval_103_rounded_avg.diff ===
      //You are given two positive integers n and m, and your task is to compute the
      // average of the integers from n through m (including n and m). 
      // Round the answer to the nearest integer and convert that to binary.
      // If n is greater than m, return -1.
      // Example:
      // >>> rounded_avg(1, 5)
      // "0b11"
      // >>> rounded_avg(7, 5)
      // -1
      // >>> rounded_avg(10, 20)
      // "0b1111"
      // >>> rounded_avg(20, 33)
      // "0b11010"
    - function rounded_avg(n: number, m: number): string| number {
    + function rounded_avg(n: number, m: number): string | number {
    ?                                                   +
    
    === humaneval-ts_prompt_HumanEval_125_split_words.diff ===
      //Given a string of words, return an array of words split on whitespace, if no whitespaces exists in the text you
      // should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the
      // alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25
      // Examples
      // >>> split_words("Hello world!")
      // ["Hello", "world!"]
      // >>> split_words("Hello,world!")
      // ["Hello", "world!"]
      // >>> split_words("abcdef")
      // 3
    - function split_words(txt: string): string[]| number {
    + function split_words(txt: string): string[] | number {
    ?                                            +
    
    === mbpp-ts_prompt_mbpp_253_count_integer.diff ===
      //Write a tsthon function that returns the number of integer elements in a given array.
    - function count_integer(list1: number| string| number[]): number {
    + function count_integer(list1: (number | string | number)[]): number {
    ?                               +      +        +        +
    
    === mbpp-ts_prompt_mbpp_278_count_first_elements.diff ===
      //Write a function to find the number of elements that occurs before the array element in the given array.
    - function count_first_elements(test_tup: number| [number, number][]): number {
    + function count_first_elements(test_tup: (number | [number, number])[]): number {
    ?                                         +      +                  +
    
    === mbpp-ts_prompt_mbpp_294_max_val.diff ===
      //Write a function to find the maximum value in a given heterogeneous array.
    - function max_val(listval: string| number[]): number {
    + function max_val(listval: (string | number)[]): number {
    ?                           +      +        +
    
    === mbpp-ts_prompt_mbpp_297_flatten_list.diff ===
      //Write a function to flatten a given nested array structure.
    - function flatten_list(list1: number| number[][]): number[] {
    + function flatten_list(list1: (number | number[])[]): number[] {
    ?                              +      +          +
    
    === mbpp-ts_prompt_mbpp_405_check_tuplex.diff ===
      //Write a function to check whether an element exists within an array.
    - function check_tuplex(tuplex: string| number[], tuple1: any): boolean {
    + function check_tuplex(tuplex: (string | number)[], tuple1: any): boolean {
    ?                               +      +        +
    
    === mbpp-ts_prompt_mbpp_410_min_val.diff ===
      //Write a function to find the minimum value in a given heterogeneous array.
    - function min_val(listval: string| number[]): number {
    + function min_val(listval: (string | number)[]): number {
    ?                           +      +        +
    
    === mbpp-ts_prompt_mbpp_419_round_and_sum.diff ===
      //Write a function to round every number of a given array of numbers and print the total sum multiplied by the length of the array.
    - function round_and_sum(list1: number| number[]): number {
    + function round_and_sum(list1: (number | number)[]): number {
    ?                               +      +        +
    
    === mbpp-ts_prompt_mbpp_65_recursive_list_sum.diff ===
      //Write a function to flatten an array and sum all of its elements.
    - function recursive_list_sum(data_list: number| number[][]): number {
    + function recursive_list_sum(data_list: (number | number[])[]): number {
    ?                                        +      +          +
    
    === mbpp-ts_prompt_mbpp_755_second_smallest.diff ===
      //Write a function to find the second smallest number in an array.
    - function second_smallest(numbers: number| number[]): number | undefined {
    + function second_smallest(numbers: (number | number)[]): number | undefined {
    ?                                   +      +        +
    

    I'll get this out with a new release soon.

  5. nielstron commented on Nov 13, 2024

    @nielstron
    Author

    Great. Btw I also noticed some issues regarding replacing "py" with "ts" leading to eg "tsthon" in prompts.

  6. arjunguha commented on Nov 14, 2024

    @arjunguha
    Member

    Yeah, we've noticed that too. I doubt that has much of an impact so I wasn't planning to fix it.

  7. added a commit that references this issue on Nov 18, 2024
  8. arjunguha commented on Nov 18, 2024

    @arjunguha
    Member

    These are now pushed to the hub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions