Repository navigation
Proposal: Variadic Kinds -- Give specific types to variadic functions #5453
Description
Activity
- addedSuggestionAn idea for TypeScriptAn idea for TypeScriptIn DiscussionNot yet reached consensusNot yet reached consensus
on Oct 29, 2015 +1, this is really useful for functional programming in TypeScript! How would this work with optional or rest arguments? More concrete, can the
composefunction be used on functions with rest arguments or optional arguments?Reacted by Arlair, Timofey Kachalov, A. Matías Quezada, Lukas, Igor Soloydenko, Code Scratcher, Edward Bramanti, Roman Tezikov, Martin Chaov, Khải and 25 moresandersn commented
on Nov 2, 2015 MemberAuthorMore actionsGood point. I think you could assign the smallest allowed tuple type to an optional-param function since tuple are just objects, which allow additional members. But that's not ideal. I'll see if I can figure out the
composeexample and then I'll update the proposal.sandersn commented
on Nov 2, 2015 MemberAuthorMore actionsActually union types would probably work better. Something like
function f(a: string, b? number, ...c: boolean[]): number; function id<T>(t: T): T; let g = compose(f, id): (...ts: ([string] | [string, number] | [string, number, boolean[]]) => number g("foo"); // ok g("foo", 12); // ok g("foo", 12, [true, false, true]); // ok
This still breaks rest parameters, though.
Reacted by Paul Marlow, Vsevolod Trofimov, Joram Ruitenschild and Krzysztof Krzeszewskisandersn commented
on Nov 4, 2015 MemberAuthorMore actionsAnders Hejlsberg (@ahejlsberg), you had some ideas how tuple kinds would work, I think.
So 👍 on this. For information this is related to (and would fulfill) #3870. We have tried to implement a compose type API in TypeScript but are having to work around some of the limitations noted in this proposal. This would certainly solve some of those problems!
It seems though that sometimes you may want to "merge" such tuple types instead of persisting them, especially with something like compose. For example:
function compose<T, ...U>(base: T, ...mixins: ...U): T&U { /* mixin magic */ }
Also, in a lot of your examples, you have been using primitives. How would you see something more complex working, especially if there are conflicts?
Reacted by rzvc, Roy Tinker, smqz, Paul Marlow, Tingan Ho, Eduard Dykman, Nenad Vićentić, Matthew Dean, prescientmoon, Jordan and 1 moresandersn commented
on Nov 5, 2015 MemberAuthorMore actionsUnfortunately this proposal as-is does not address #3870 or the type composition, since the only composition operator for tuple kinds is
[T,...U]. You could also write this asT + ...U(which is more indicative of what happens to the types), but #3870 and your type composition library needT & ...U. I think that might be possible, but I need to understand Jason Freeman (@JsonFreeman)'s and Jon (@jbondc)'s ideas from #3870 first. I'll expand the proposal if I can figure out how it should work.Note: I decided to go with the syntax
[...T, ...U]because it looks like the equivalent value spreading syntax, butT + ...Uis more indicative of what's happening with the types. If we end up with both, then+and&might be the operators to use.Big 👍 on this!
+1 awesome! It would allow to express such things much more expressive and lightweight.
JsonFreeman commented
on Nov 6, 2015 ContributorMore actionsMy point in #3870 seems to be an issue here. Specifically, I worry about inferring type arguments for variadic type parameters.
Type argument inference is a rather complicated process, and it has changed in subtle ways over time. When arguments are matched against parameters in order to infer type type arguments, there are no guarantees about the order in which candidates are inferred, nor how many candidates are inferred (for a given type parameter). This has generally not been a problem because the result surfaced to the user does not (in most cases) expose these details. But if you make a tuple type out of the inference results, it certainly does expose both the order and the count of the inferences. These details were not intended to be observable.
How serious is this? I think it depends on how exactly the inference works. What is the result of the following:
function f<...T>(x: ...T, y: ...T): ...T { } f(['hello', 0, true], [[], 'hello', { }]); // what is the type returned by f?
sandersn commented
on Nov 6, 2015 MemberAuthorMore actionsJon (@jbondc),
-seems like a good idea. I'll keep it in mind but not explore it here, because I think we should introduce new type operators one at a time. Both&and+create new types, but&creates an intersection type whereas+creates a new tuple type (which is why I prefer the syntax[T,...U]instead ofT + ...U, because[T,U]already does this for types).Jason Freeman (@JsonFreeman) l think it's OK to do one of two things with repeated kind parameters:
- Union the types:
f(['hello', 1], [1, false]): [string | number, number | boolean] - Disallow inference of repeated tuple kind parameters, particularly if type argument inference proves complicated. Something like this:
f(['hello', 1], [1, false]) // error, type arguments required f<[string, number]>(['hello', 1], [1, false]) // error, 'number' is not assignable to 'string' f<[string | number, number | boolean]>(['hello', 1], [1, false]); // ok
I think real libraries (like the reactive extensions Igor Oleinikov (@Igorbek) linked to) will usually only have one tuple kind parameter so even though neither (1) nor (2) are particularly usable, it shouldn't impact real-world code much.
In the examples above,
curryis the hardest to infer -- you have to skipf: (...args:[...T,...U]) => V, infer...ts:...T, then go back and set...Uto what's left after consuming...Tfromf's parameters.I've started prototyping this (sandersn/TypeScript@1d5725d), but haven't got that far yet. Any idea if that will work?
- Union the types:
JsonFreeman commented
on Nov 6, 2015 ContributorMore actionsI would err on the side of disallowing anything where the semantics is not clear (like repeated inferences to the same spreaded type parameter). That allays my concern above as well.
I can't think of a good mechanism for typing curry. As you point out, you have to skip the parameter list of the first function to consume the
...Targument and then see what's left over. There would have to be some policy to postpone inferences to a spreaded type parameter if it's not final in its list. It could get messy.That said, I think this is worth a try. There is high demand for the feature.
sandersn commented
on Nov 6, 2015 MemberAuthorMore actionsI think you would have to skip multiple tuple kinds that occur in the same context (eg top-level like
(...T,string,...U) => Vor concatenated like[...T,...U,...T]). Then you can make multiple passes on the skipped kinds, eliminating already-inferred kinds and re-skipping kinds that are still ambiguous. If at any point no single kind is available for inference, stop and return an error.So, yeah. Complicated.
JsonFreeman commented
on Nov 7, 2015 ContributorMore actionsYou may be able to draw inspiration from a similar problem. It is actually somewhat similar to the problem of inferring to a union or intersection. When inferring to a union type that includes a type parameter that is a member of the inference context, as in
function f<T>(x: T | string[]), you don't know whether to infer to T. The intended manifestation of the union type may have beenstring[]. So typescript first infers to all other constituents, and then if no inferences were made, infers to T.In the case of intersection, it's even harder because you may have to split the type of the argument across the different intersection constituents. Typescript doesn't make inferences to intersection types at all.
What if you only allowed spreading tuple if it is the last type in its sequence? So
[string, ...T]would be allowed, but[...T, string]would not be?Reacted by Dave Houlbrooke246 remaining items
Load more actionsI don't believe this PR solves this particular issue tbh.
With the PR this works
function foo<T extends any[]>(a: [...T]) { console.log(a) } foo<[number, string]>([12, '13']);
But this issue would like to see an implementation for this as far as I see:
function bar<...T>(...b: ...T) { console.log(b) } bar<number, string>(12, '13');
There is a lot angle brackets there, looks a little redundant.
Reacted by Alexey Iskhakov, Jarrettgxz and Yury MatuhinSandor (@AlexAegis) I'm not sure I see a lot of value in "rest type parameters" like that. You can already do this:
declare function foo<T extends any[]>(...a: T): void; foo(12, '13'); // Just have inference figure it out foo<[number, string]>(12, '13'); // Expclitly, but no need to
Don't think we really want a whole new concept (i.e. rest type parameters) just so the square brackets can be avoided in the rare cases where inference can't figure it out.
Reacted by Trey Brisbane, Karel Hrkal, Mahmoud Samy, Jarrettgxz and François LevasseurAnders Hejlsberg (@ahejlsberg) I see. I was asking because some libraries (RxJS as mentioned) used workarounds to provide this functionality. But it's finite.
bar<T1>(t1: T1); bar<T1, T2>(t1: T1, t2:T2); bar<T1, T2, T3>(t1: T1, t2:T2, t3: T3, ...t: unknown) { ... }
So now they either stick with that, or have the users type the brackets, which is a breaking change, and not that intuitive.
The reason why I used this example is because here it's straightforward that I defined the type of that tuple. One square bracket here, one there
foo<[number, string]>([12, '13']);
Here it's not so obvious that the tuple refers to that rest parameter if you look at it from the outside
foo<[number, string]>(12, '13');
But yes as you said if we let the inference figure it out then these trivial cases are not requiring any modification from the user. But we don't know if they did set them explicitly or not, it's up to them, so it still counts as a breaking change. But that's the lib's concern and not this change's.
That said I just find it odd that if there are rest parameters, defined from the outside one by one, that are a single array on the inside differentiated by
..., cannot be made generic the same way: one by one on the outside, single array on the inside, differentiated by....Reacted by András Tóth- Minor syntax discrepancies are not really worth support cost for a separate kind. Using kinds would be a correct design decision when TS was planning support for rest parameters, but I guess now it might lead to more confusion both for language developers and users. We needed a solution for this issue, and Anders did his job exceptionally well avoiding that complexity by sticking to `[...T]` instead of `T`. Hats off! (Could we now take a look on a bug that unifying intersection type to inferred variable in conditional type returns rightmost intersection type argument, or that union of arrays is not array of union please? We still have major showstoppers in type system.)…On Fri, Jun 19, 2020, 10:41 Győri Sándor ***@***.***> wrote: @ahejlsberg <https://github.com/ahejlsberg> I see. I was asking because some libraries (RxJS as mentioned) used workarounds to provide this functionality. But it's finite. bar<T1>(t1: T1);bar<T1, T2>(t1: T1, t2:T2);bar<T1, T2, T3>(t1: T1, t2:T2, t3: T3, ...t: unknown) { ... } So now they either stick with that, or have the users type the brackets, which is not that intuitive. The reason why I used this example is because here it's straightforward that I defined the type of that tuple. One square bracket here, one there foo<[number, string]>([12, '13']); Here it's not so obvious that the tuple refers to that rest parameter if you look at it from the outside foo<[number, string]>(12, '13'); — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5453 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWYQIMTTB6JEPSQFUMTMDTRXMJD5ANCNFSM4BTBQ7DQ> .
I am of course nowhere close to his caliber, but I respectfully disagree with Anders Hejlsberg (@ahejlsberg) .
In my experience, much of the complexity of typescript comes from the fact that a lot of (interesting and useful to be sure) features are special-cased in as their own concepts.This complexity is not inherently a function of the number of features though!
Instead, the language could be designed around larger, more overarching concepts from which these special cases could then be trivially deduced, or implemented in the std (type) library.The most general such concept would of course be to fully implement dependent types, from which everything else could then be derived, but going that far is not necessary:
As C++ and, to a lesser extent, Rust have shown, a few large scale, consistent concepts give you a ton of features for free.
This is similar to what OCaml and Haskell (and I assume F#?) have done on the value level, just on the type level.Type level programming is nothing to be scared of as long as it is designed into the language instead of tacked on to provide specific features.
The facilities in C++ 14/17 are very intuitive except for their syntax, which is purely due to historical baggage.Reacted by Sandor, Jason Kuhrt and Aleksey Gurtovoy- Overarching concepts could have been added in original design. After design mistake was already made, consistency cannot be added without risking huge back-incompatibility. I agree with suspicions regarding language design as a whole (TS is quite far from standards set by academia, nobody can disagree with that). There is a lot of bugs and inconsistencies that are foundational to millions of production code bases. Mere fact that developers are able to come up with useful additions to the language without accidentally fixing those bugs is, in my humble opinion, awesome and deserves respect. TS has same design complexities as C++ here, but its expressive type system makes the situation worse.…On Fri, Jun 19, 2020, 12:47 Bennett Piater ***@***.***> wrote: I am of course nowhere close to his caliber, but I respectfully disagree with @ahejlsberg <https://github.com/ahejlsberg> . *In my experience, much of the complexity of typescript comes from the fact that a lot of* (interesting and useful to be sure) *features are special-cased in as their own concepts.* This complexity is not inherently a function of the number of features though! Instead, the language could be designed around larger, more overarching concepts from which these special cases could then be trivially deduced, or implemented in the std (type) library. The most general such concept would of course be to fully implement dependent types, from which everything else could then be derived, but going that far is not necessary: As C++ and, to a lesser extent, Rust have shown, a few large scale, consistent concepts give you a ton of features for free. This is similar to what OCaml and Haskell (and I assume F#?) have done on the value level, just on the type level. Type level programming is nothing to be scared of as long as it is designed into the language instead of tacked on to provide specific features. The facilities in C++ 14/17 are very intuitive except for their syntax, which is purely due to historical baggage. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5453 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWYQIMWYLGGCWPTDBZJR4TRXMX4RANCNFSM4BTBQ7DQ> .Reacted by Bennett Piater
@polkovnikov-ph I'm glad we agree on the issue at hand :)
As for the solution, I think it would still be worth considering progressively moving towards a more carefully designed type system. Major versions are a thing after all, and the alternative is to end up in the cluster**** that is C++ 20 - an admirable attempt at adding even more nicely designed features on top of 2 layers of previous attempts that cannot be removed, in a syntax that is already not deterministically parseable.
Reacted by Giora Guttsait, Jason Kuhrt, reverofevil, Homa Wong and Flávio LisbôaAll of this is off-topic to this thread and is being discussed here. So I'll try to be frank:
It took decades for academia to figure out correct approach to subtyping: mlsub type system was created only 6 years ago, well after TypeScript was first released. It could be that foundation for classes, interfaces, union and intersection types with overarching concepts.
But also remember there are conditional types. I'm not aware of any papers giving them formal semantics, or describing a minimal type system with conditional types with progress/preservation proofs. I believe that might have something to do with scientists still being shy to print their failed attempts. If your proposal assumes those major incompatible versions will be made in 2040's, when academia gets comfortable with conditional types, I can agree.
Otherwise "carefully designed type system" would have to remove conditional types from the language, and I don't think anyone is up to the task of converting 60% of DefinitelyTyped to use whatever alternative is chosen to replace them. (And then do it several more times, because it's not the only issue.)
I'm afraid the only viable solution is to create a separate programming language that would somehow resemble TS, and somehow (not only by being more pleasurable to write code in) lure developers to use it. Ryan was quite vocal recommending this approach for TS improvement previously.
Reacted by Sandor and Felipe S. S. SchneiderCould this be used to improve the type definitions for
Array.prototype.map? Right now it returnsT[]even if tuple like[T, T]was passed in.Reacted by ExE Boss, Shad Sterling, Toni Villena, brunnerh, Slim and Mingwei SamuelReacted by Bo Lingen and Toni Villena@kevinbarabash You can already do that with overloads:
declare global { interface Array<T> { map<U>(this: T[] | [T], callbackfn: (value: T, index: number, array: this) => U, thisArg?: any): { [index in keyof this]: U }; } } const input: [number, number] = [1, 2]; const output = input.map(x => x + 1); // [number, number]
A similar overload can be added to
ReadonlyArrayso that it works with tuples created withas const.Reacted by Daniele Orlando, Vitor L Cavalcanti, Shad Sterling and slooiReacted by Kevin BarabashKarol Majewski (@karol-majewski) thank you. I'm surprised that that snippet handles tuples of different lengths. I need to do some reading on
this: T[] | [T]because I'd expect it only to handle 1-tuples.Reacted by Shad Sterling, Bo Lingen and Pedro Augusto de Paula BarbosaSee also: #36554
Tylor Reynolds (@tylorr) Doesn't quite work, due to a circular type error.
However, the usual workaround works.
type Last<T extends readonly unknown[]> = T extends readonly [...infer _, infer U] ? U : undefined; interface UnaryFunction<T, R> { (source: T): R; } type PipeParams<T, R extends unknown[]> = { 0: [], 1: R extends readonly [infer U, ...infer V] ? [UnaryFunction<T, U>, ...PipeParams<U, V>] : never }[R extends readonly [unknown] ? 1 : 0]; declare function pipe<T, R extends unknown[]>(...fns: PipeParams<T, R>): UnaryFunction<T, Last<R>>;
Interestingly, this doesn't work either, I've added a simple example in the playground that fails.
I've got to something similar independently and was surprised that it didn't work :(Does anyone know what's the way to type
pipein TSv4.5?
Variadic Kinds
Give Specific Types to Variadic Functions
This proposal lets Typescript give types to higher-order functions that take a variable number of parameters.
Functions like this include
concat,apply,curry,composeand almost any decorator that wraps a function.In Javascript, these higher-order functions are expected to accept variadic functionsas arguments.
With the ES2015 and ES2017 standards, this use will become even more common as programmers start using spread arguments and rest parameters for both arrays and objects.
This proposal addresses these use cases with a single, very general typing strategy based on higher-order kinds.
This proposal would completely or partially address several issues, including:
I'll be updating this proposal on my fork of the Typescript-Handbook: sandersn/TypeScript-Handbook@76f5a75
I have an in-progress implementation at sandersn/TypeScript@f3c327a which currently has the simple parts of the proposal implemented.
It supercedes part 2 of my previous proposal, #5296.
Edit: Added a section on assignability. I'm no longer sure that it strictly supercedes #5296.
Preview example with
currycurryfor functions with two arguments is simple to write in Javascript and Typescript:and in Typescript with type annotations:
However, a variadic version is easy to write in Javascript but cannot be given a type in TypeScript:
Here's an example of using variadic kinds from this proposal to type
curry:The syntax for variadic tuple types that I use here matches the spread and rest syntax used for values in Javascript.
This is easier to learn but might make it harder to distinguish type annotations from value expressions.
Similarly, the syntax for concatenating looks like tuple construction, even though it's really concatenation of two tuple types.
Now let's look at an example call to
curry:In the first call,
In the second call,
Syntax
The syntax of a variadic kind variable is
...Twhere T is an identifier that is by convention a single upper-case letter, orTfollowed by aPascalCaseidentifier.Variadic kind variables can be used in a number of syntactic contexts:
Variadic kinds can be bound in the usual location for type parameter binding, including functions and classes:
And they can be referenced in any type annotation location:
Variadic kind variables, like type variables, are quite opaque.
They do have one operation, unlike type variables.
They can be concatenated with other kinds or with actual tuples.
The syntax used for this is identical to the tuple-spreading syntax, but in type annotation location:
Tuple types are instances of variadic kinds, so they continue to appear wherever type annotations were previously allowed:
Semantics
A variadic kind variable represents a tuple type of any length.
Since it represents a set of types, we use the term 'kind' to refer to it, following its use in type theory.
Because the set of types it represents is tuples of any length, we qualify 'kind' with 'variadic'.
Therefore, declaring a variable of variadic tuple kind allows it to take on any single tuple type.
Like type variables, kind variables can only be declared as parameters to functions, classes, etc, which then allows them to be used inside the body:
Calling a function with arguments typed as a variadic kind will assign a specific tuple type to the kind:
Assigns the tuple type
...T=[number,number,string]...T. So in this application off,let a:...Tis instantiated aslet a:[number,number,string]. However, because the type ofais not known when the function is written, the elements of the tuple cannot be referenced in the body of the function. Only creating a new tuple froma` is allowed.For example, new elements can be added to the tuple:
Like type variables, variadic kind variables can usually be inferred.
The calls to
conscould have been annotated:For example,
consmust infer two variables, a type H and a kind ...Tail.In the innermost call,
cons("foo", ["baz", false]),H=stringand...Tail=[string,boolean].In the outermost call,
H=numberand...Tail=[string, string, boolean].The types assigned to ...Tail are obtained by typing list literals as tuples -- variables of a tuple type can also be used:
Additionally, variadic kind variables can be inferred when concatenated with types:
Here, the type of
lis inferred as[number, string, boolean].Then
H=numberand...Tail=[string, boolean].Limits on type inference
Concatenated kinds cannot be inferred because the checker cannot guess where the boundary between two kinds should be:
The checker cannot decide whether to assign
...T = [string,string,string], ...U = [string]...T = [string,string], ...U = [string,string]...T = [string], ...U = [string,string,string]Some unambiguous calls are a casualty of this restriction:
The solution is to add type annotations:
Uncheckable dependencies between type arguments and the function body can arise, as in
rotate:This function can be typed, but there is a dependency between
nand the kind variables:n === ...T.lengthmust be true for the type to be correct.I'm not sure whether this is code that should actually be allowed.
Semantics on classes and interfaces
The semantics are the same on classes and interfaces.
TODO: There are probably some class-specific wrinkles in the semantics.
Assignability between tuples and parameter lists
Tuple kinds can be used to give a type to rest arguments of functions inside their scope:
In this example, the parameter list of
f: (a: number, b:string) => stringmust be assignable to the tuple type instantiated for the kind...T.The tuple type that is inferred is
[number, string], which means that(a: number, b: string) => stringmust be assignable to(...args: [number, string]) => string.As a side effect, function calls will be able to take advantage of this assignability by spreading tuples into rest parameters, even if the function doesn't have a tuple kind:
Tuple types generated for optional and rest parameters
Since tuples can't represent optional parameters directly, when a function is assigned to a function parameter that is typed by a tuple kind, the generated tuple type is a union of tuple types.
Look at the type of
hafter it has been curried:Here
...T=([number] | [number, string]), socurried: ...([number] | [number, string]) => numberwhich can be called as you would expect. Unfortunately, this strategy does not work for rest parameters. These just get turned into arrays:Here,
curried: ...([string, boolean[]] | [boolean[]]) => number.I think this could be supported if there were a special case for functions with a tuple rest parameter, where the last element of the tuple is an array.
In that case the function call would allow extra arguments of the correct type to match the array.
However, that seems too complex to be worthwhile.
Extensions to the other parts of typescript
However, this proposal requires variadic kinds to be bindable to a empty tuple.
So Typescript will need to support empty tuples, even if only internally.
Examples
Most of these examples are possible as fixed-argument functions in current Typescript, but with this proposal they can be written as variadic.
Some, like
consandconcat, can be written for homogeneous arrays in current Typescript but can now be written for heteregoneous tuples using tuple kinds.This follows typical Javascript practise more closely.
Return a concatenated type
Concatenated type as parameter
Variadic functions as arguments
TODO: Could
freturn...Uinstead ofU?Decorators
Open questions
bind,callandapplyare methods defined on Function, their type arguments need to be bound at function-creation time rather than thebindcall site (for example). But this means that functions with overloads can't take or return types specific to their arguments -- they have to be a union of the overload types. Additionally, Function doesn't have a constructor that specifies type arguments directly, so there's really no way provide the correct types tobindet al. TODO: Add an example here. Note that this problem isn't necessarily unique to variadic functions.