PEP 750 – Template Strings(peps.python.org) |
PEP 750 – Template Strings(peps.python.org) |
I think this gives you slightly more control before interpolating.
If you want control flow inside a template, jinja and friends are probably still useful.
It is now be a generic expression evaluator and a template rendered!
> https://peps.python.org/pep-0750/#approaches-to-lazy-evaluat...
Hmm, I have a feeling there's a pitfall.
Excited to see what libraries and tooling comes out of this.
This is one place where s-expressions of Lisp make embedding these DSLs syntactically easier.
To borrow the PEP's HTML examples:
#lang racket/base
(require html-template)
(define evil "<script>alert('evil')</script>")
(html-template (p (% evil)))
; <p><script>alert('evil')</script></p>
(define attributes '((src "shrubbery.jpg") (alt "looks nice")))
(html-template (img (@ (%sxml attributes))))
; <img src="shrubbery.jpg" alt="looks nice">
PEP 750 t-strings literals work with python's tripe-quote syntax (and its lesser-used implicit string concat syntax):
lots_of_html = t"""
<div>
<main>
<h1>Hello</h1>
</main>
</div>
"""
When concatenating strings is the harder approach, it is really beautiful.
>>> hello_world = {"hello":"HELL" ,"world":"O'WORLD"}
>>> json_template='{"hello":"%(hello)s","world":"%(world)s"}'
>>> print(json_template % hello_world)
{"hello":"HELL","world":"O'WORLD"}
I mostly use Python in scientific contexts, and hitting end-of-life after five years means that for a lot project, code needs to transition language versions in the middle of a project. Not to mention the damage to reproducibility. Once something is marked "end of life" it means that future OS versions are going to have a really good reason to say "this code shouldn't even be able to run on our new OS."
Template strings seem OK, but I would give up all new language features in a heartbeat to get a bit of long term support.
> There should be one-- and preferably only one --obvious way to do it.
Use f-strings if you can, otherwise use t-strings.
I'm really loving this lovecraftian space the "batteries included" and "one obvious way to do it" design philosophy brought us!
The stated use case is to avoid injection attacks. However the primary reason why injection attacks work is that the easiest way to write the code makes it vulnerable to injection attacks. This remains true, and so injection attacks will continue to happen.
Templates offer to improve this by adding interpolations, which are able to do things like escaping. However the code for said interpolations is now located at some distance from the template. You therefore get code that locally looks good, even if it has security mistakes. Instead of one source of error - the developer interpolated - you now have three. The developer forgot to interpolate, the developer chose the wrong interpolation, or the interpolation itself got it wrong. We now have more sources of error, and more action at a distance. Which makes it harder to audit the code for sources of potential error.
This is something I've observed over my life. Developers don't notice the cognitive overhead of all of the abstractions that they have internalized. Therefore over time they add more. This results in code that works "by magic". And serious problems if the magic doesn't quite work in the way that developers are relying on.
Templates are yet another step towards "more magic". With predictable consequences down the road.
Background: TXR already Lisp has quasi-string-literals, which are template strings that do implicit interpolation when evaluated. They do not produce an object where you can inspect the values and fixed strings and do things with these before the merge.
1> (let ((user "Bob") (greeting "how are you?"))
`Hello @user, @greeting`)
"Hello Bob, how are you?"
2> (car '`Hello @user, @greeting`)
sys:quasi
3> (cdr '`Hello @user, @greeting`)
("Hello " @user ", " @greeting)
OK, now with those pieces, I just right now made a macro te that gives us a template object.
4> (load "template")
nil
5> (let ((user "Bob") (greeting "how are you?"))
(te `Hello @user, @greeting`))
#S(template merge #<interpreted fun: lambda (#:self-0073)> strings #("Hello " ", ")
vals #("Bob" "how are you?"))
6> *5.vals
#("Bob" "how are you?")
7> *5.strings
#("Hello " ", ")
8> *5.(merge)
"Hello Bob, how are you?"
9> (set [*5.vals 0] "Alice")
"Alice"
10> *5.(merge)
"Hello Alice, how are you?"
(We cannot alter the strings in this implementation; they are for "informational purposes only").
Here is how the macro expands:
11> (macroexpand-1 '(te `Hello @user, @greeting`))
(new template
merge (lambda (#:self-0073)
(let* ((#:vals-0074
#:self-0073.vals)
(#:var-0075
[#:vals-0074
0])
(#:var-0076
[#:vals-0074
1]))
`Hello @{#:var-0075}, @{#:var-0076}`))
strings '#("Hello " ", ")
vals (vec user greeting))
The initialization of strings is trivial: just a vector of the strings pulled from the quasistring.
The vals slot is initialized by a `(vec ...)` call whose arguments are the expressions from the quasistring. This gets evaluated in the right lexical scope where the macro is expanded. This is how we capture those values.
The most complicated part is the lambda expression that initializes merge. This takes a single argument, which is the self-object, anonymized by a gensym variable for hygiene. It binds the .vals slot of the object to another gensym lexical. Then a genyms local variable is bound for each value, referencing into consecutive elements of the value vector. E.g. #:var-0075 is bound to [#:vals-0074 0], the first value.
The body of the let is a transformed version of the original template, in which the interpolated expressions are replaced by gensyms, which reference the bindings that index into the vector.
The complete implementation in template.tl (referenced by (load "template") in command line 4) is:
(defstruct template ()
merge
strings
vals)
(defun compile-template (quasi)
(match (@(eq 'sys:quasi) . @args) quasi
(let ((gensyms (build-list))
(exprs (build-list))
(strings (build-list))
(xquasi (build-list '(sys:quasi)))
(self (gensym "self-"))
(vals (gensym "vals-")))
(while-true-match-case (pop args)
((@(eq 'sys:var) @(bindable @sym))
exprs.(add sym)
(let ((g (gensym "var-")))
gensyms.(add g)
xquasi.(add g)))
((@(eq 'sys:expr) @expr)
exprs.(add expr)
(let ((g (gensym "expr-")))
gensyms.(add g)
xquasi.(add g)))
(@(stringp @str)
strings.(add str)
xquasi.(add str))
(@else (compile-error quasi
"invalid expression in template: ~s" else)))
^(new template
merge (lambda (,self)
(let* ((,vals (qref ,self vals))
,*[map (ret ^(,@1 [,vals ,@2])) gensyms.(get) 0])
,xquasi.(get)))
strings ',(vec-list strings.(get))
vals (vec ,*exprs.(get))))))
(defmacro te (quasi)
(compile-template quasi))
That Lisp Curse document, though off the mark in general, was right the observation that social problems in languages like Python are just technical problems in Lisp (and often minor ones).
In Python you have to wait for some new PEP to be approved in order to get something that is like f-strings but gives you an object which intercepts the interpolation. Several proposals are tendered and then one is picked, etc. People waste their time producing rejected proposals, and time on all the bureucracy in general.
In Lisp land, oh we have basic template strings already, let's make template objects in 15 minutes. Nobody else has to approve it or like it. It will backport into older versions of the language easily.
P.S.
I was going to have the template object carry a hash of those values that are produced by variables; while coding this, I forgot. If we know that an interpolation is @greeting, we'd like to be access something using the greeting symbol as a key.
(I don't see any of this is as useful, so I don't plan on doing anything more to it. It has no place in Lisp, because for instance, we would not take anything resembling this approach for HTML generation, or anything else.)
And your scientific context is a distinct minority for python now. Most new development for python is for data/AI. Considering LLMs get updated every quarter, and depreciated every year, there is no appetite for code that doesn't get updated for 5 years.
The code will be updated over five years, but there's no need to be on continual version churn on the underlying language. And frankly I'm surprised that it's tolerated so widely in the community. Trying to run a Node project from 5 years ago is often an exercise in futility, and it will be a big shame when/if that happens to Python.
Your Python interpreter will not spontaneously combust due to being end-of-life. It just eventually won't be able to run new versions of tools; but your existing tool versions should also work fine for a very long time. All you're missing out on is bugfixes, which third parties (such as a Linux distro) are often willing to provide.
When a language like Python doesn't innovate at this rate, eventually people will get annoyed about how primitive it ends up feeling compared to languages that have been around for less than half as long. The devs' time and resources are limited and they've clearly advertised their time preference and committed to a reliable schedule - this is an explicit attempt to accommodate users like you better, compared to the historical attitude of releasing the next minor version "when it's done". It also means that they're locked in to supporting five versions at a time while developing a sixth. There's only so much that can reasonably be expected here.
Seriously, what you're getting here is well above the curve for open-source development.
But it's not a long time in the OP's field of science. Unfortunately despite a strong preference for Python in the scientific community, the language's design team seem to ignore that community's needs entirely, in favour of the needs of large technology companies.
I was hopeful that in the transition from a BDFL-based governance system to a Steering Council, we would see a larger variety of experience and opinions designing the language. Instead, I don't think there has ever been a single scientist, finance worker etc on the Steering Council - it's always software developers, almost always employees of large software companies.
Template.__str__() is equivalent to Template.__repr__(), which is to say that these aren't f-strings in an important sense: you can't get a useful string out of them until you process them in some way.
The expectation is that developers will typically make use of well-established libraries that build on top of t-strings. For instance, developers might grab a package that provides an html() function that accepts Template instances and returns some Element type, which can then be safely converted into a string.
Stepping back, t-strings are a pythonic parallel to JavaScript's tagged template strings. They have many of the same advantages and drawbacks.
In PHP, people used to just call mysql_query on a string and all the escaping was done with mysql_escape_string. According to you that nice locality of query construction and sanitization that should've improved security, but my god did it ever not do that.
It was exactly layers of abstractions, moving things far away from the programmer, with prepared statements to ORMs, that meaningfully reduced the number of SQL injection vulnerabilities.
Another example is JavaScript, how many XSS vulnerabilities never happened because of all the shadow dom frameworks? Layers of abstractions like these (JSX,etc) are a major reason we don't see many XSS vulnerabilities nowadays.
The idea is for the interpolation to be provided by the library - just as the library is expected to provide a quoting/escaping/sanitization function today. But now the interpolation function can demand to receive an instance of the new Template type, and raise a `TypeError` if given a pre-formatted string. And that work can perhaps also be rolled into the same interface as the actual querying command. And manually creating a Template instance from a pre-formatted string is difficult and sticks out like a sore thumb (and it would be easy for linters to detect the pattern).
> This is something I've observed over my life. Developers don't notice the cognitive overhead of all of the abstractions that they have internalized. Therefore over time they add more. This results in code that works "by magic". And serious problems if the magic doesn't quite work in the way that developers are relying on.
By this logic, we couldn't have languages like Python at all.
Could you give examples of this?
> The developer forgot to interpolate
What would this look like? The only way to get dynamic/user input into a template is either through interpolation or concatenation.
Before:
f"..html_str..." + user_provided_str # oops! should have: html_str + sanitize(user_provided_str)
After:
t"...html_template..." + user_provided_str # oops! should have: t"...html_template...{user_provided_str}"
Does this really leave us worse off?
Unless you're referring to something like this:
Before:
html = "love > war" # oops! should have been: html = "love > war"
After:
html = "love > war" # oops! should have been: html = t"love > war"
But then the two scenarios are nearly identical.
> the developer chose the wrong interpolation
What kind of interpolation would be the "wrong interpolation"?
> or the interpolation itself got it wrong.
Isn't that analogous to sanitize(user_provided_str) having a bug?
> the developer chose the wrong interpolation Not possible if the library converts from template to interpolation itself
> or the interpolation itself got it wrong Sure, but that would be library code.
Just this week I had difficulty integrating the work of a team member because they used some new typing features only available in Python 3.13, but we have many library dependencies on numpy < 2, and in their great wisdom somebody decided that with Python 3.13 there would be no more precompiled wheels of numpy < 2. Meaning arduous multiple-minute compilation for any new venv or Docker build, even with uv. This sort of pointless version churn, wasting many valuable research hours on investigating the chains of dependencies and which libraries are ready or not, to serve the whims of some software engineer that decides everyone must update working code to novel APIs, is not something that I experience in other languages.
Hopefully Python Steering Council members reconsider the motivation of continual churn, but it's much harder to get promoted and get acknowledgement for judicious tending of a language than it is to ship a bunch of new features. Combined with fear over Anaconda charges, Python is quickly becoming a very unfriendly place for science, or anybody else that values function over form.
Wow. NumPy 2.0 was released less than a year ago and they're already starting to drop support for 1.x?! It only supports Python versions up to 3.12, all of which are themselves either unsupported or in "security fixes only" mode.
The developers plan to end-of-life 1.x entirely in less than six months time. Do they not know their target audience at all?
> it's much harder to get promoted and get acknowledgement for judicious tending of a language than it is to ship a bunch of new features
Yes - as Mark Lutz (author of "Learning Python") puts it [0]: "[Python's] evolution is largely driven by narcissism, not user feedback" and "Python remains a constantly morphing sandbox of ideas, which is great if you just want to play in the sandbox, but lousy if you're trying to engineer reliable and durable software".
[0] https://learning-python.com/python-changes-2014-plus.html#se...
(Or, in a sufficiently working program, an editor with semantic analysis access could use something like type inference in the Python side, to determine the language in a less-kludgey way.)
Yeah, something we spent a bunch of time considering. In the end, we decided it probably needed to stay out of scope for the PEP.
You're right that JavaScript has an easier time here. Most of the JS tools we looked at simply inspect the name of the tag and if it's (say) html, they attempt to color/format string content as HTML regardless of what the html() function actually does or the string's contents.
Currently, tools like black have no knowledge of types. I'm guessing some amount of kludging is to be expected on day one. But my hope is over the long term, we'll see a story emerge for how annotations can indicate the expected content type.
lots_of_html = t"""
<div>
<main>
<p>{evil}>/p>
<main>
</span>
"""It's a template instance that still needs to be safely processed into a renderable string, e.g. by escaping whatever `evil` evaluates to and even validating the final HTML syntax.
And why would you be validating HTML on the fly, when it's coming from your program, not as an input into it. Even if you can do it at program startup once for each template, it's still pointless overhead.
The whole thing is wrongheaded; exactly the kind of stove-pipe people end up inventing when they don't have metaprogramming.
No, that isn't how it works. The unprocessed version is not a `str` instance and doesn't implement `__str__`:
> This is because Template instances are intended to be used by template processing code, which may return a string or any other type. There is no canonical way to convert a Template to a string.
If you tried to use the Template directly as if it were a string, you'd get either a TypeError or completely malformed HTML (the `repr` of the Template instance, which would look very different).
>And why would you be validating HTML on the fly
You wouldn't be; you'd be escaping user-generated content that tries to break a page by including HTML markup.
... but let me assure you it's never the wrong one!
I’m curious how?
> The whole thing is wrongheaded; exactly the kind of stove-pipe people end up inventing when they don't have metaprogramming.
Python has many metaprogramming features. I don't think you understand this feature much less its motivation.
How else would you go about adding language support for e.g. HTML and SQL within Python?
1> (load "template")
nil
2> (let ((field "name") (table "customers"))
(te `SELECT @field FROM @table`))
#S(template merge #<interpreted fun: lambda (#:self-0073)> strings #("SELECT " " FROM ")
vals #("name" "customers"))
3> *2.(merge)
"SELECT name FROM customers"
4> [*2.vals 0]
"name"
5> (set [*2.vals 0] "id")
"id"
6> *2.(merge)
"SELECT id FROM customers"
7> (upd *2.vals (map upcase-str))
#("ID" "CUSTOMERS")
8> *2.(merge)
"SELECT ID FROM CUSTOMERS"
It's a parlor trick that doesn't have any uses. The structured log messages use case is the most promising, because it has a consumer which actually wants the interpolated pieces that it would otherwise have to hackily parse out.
I predict that Python will eventually get dedicated HTML syntax: perhaps something that uses indentation to indicate element nesting. Let's "vibe pseudocode" a sketch:
html:
div (class='main' id='1'):
p:
"paragraph text"
This is a disingenuous way to compare a LISP with a language that values syntactic convention and strives to be readable and maintainable by more than one person.
> I predict that Python will eventually get dedicated HTML syntax:
How do you assign this result to a variable? How is this any better than
content: HTML = t"<p>Hello {name}</p>"Python has to ship a new version for this; there is no way for existing installations to use the code.
I don't have to change anything in my editor setup.
So who is it that values syntactic convention?
t"abc" is a syntax errror in vast numbers of existing Python installations; how can we call it convention?
> How do you assign this result to a variable?
That's to be worked out. The "html:" fantasy construct could have a way to do that. When you think of assigning to a variable, the first thing that comes to mind is "a = b" syntax. But, look, the define construct in Python also assigns to a variable:
define fun(arg):
...
> How is it any better
It renders all of the following problems unrepresentable: mismatched angle brackets, mismatched or unclosed tags, bad attribute syntax, injection.
- Java's been trying to add f/t-strings, but its designers appear to be perfectionists to a fault, unable to accept anything that doesn't solve every single problem possible to imagine: [1].
- Go developers seem to have taken no more than 5 minutes considering the problem, then thoughtlessly discarded it: [2]. A position born from pure ignorance as far as I'm concerned.
- Python, on the other hand, has consistently put forth a balanced approach of discussing each new way of formatting strings for some time, deciding on a good enough implementation and going with it.
In the end, I find it hard to disagree with Python's approach. Its devs have been able to get value from first the best variant of sprintf in .format() since 2008, f-strings since 2016, and now t-strings.
[1]: https://news.ycombinator.com/item?id=40737095
[2]: https://github.com/golang/go/issues/34174#issuecomment-14509...
There are a million things in go that could be described this way.
Are they wrong about this issue? I think they are. There is a big difference in ergonomics between String interpolation and something like fmt.Sprintf, and the performance cost of fmt.Sprintf is non-trivial as well. But I can't say they didn't put any thought into this.
As we've seen multiple times with Go generics and error handling before, their slow progress on correcting serious usability issues with the language stem from the same basic reasons we see with recent Java features: they are just being quite perfectionist about it. And unlike Java, the Go team would not even release an experimental feature unless they feel quite good about it.
A format function that arbitrarily executes code from within a format string sounds like a complete nightmare. Log4j as an example.
The rejection's example shows how that arbitrary code within the string could instead be fixed functions outside of a string. Safer, easier for compilers and programmers; unless an 'eval' for strings is what was desired. (Offhand I've only seen eval in /scripted/ languages; go makes binaries.)
An f/t string is syntax not runtime.
Instead of
"Hello " + subject + "!"
f"Hello {subject}!"
And instead of
query(["SELECT * FROM account WHERE id = ", " AND active"], [id])
query(t"SELECT * FROM account WHERE id = {id} AND active")
When compiling, those can be lowered to simple string concatenation, just like any for loop can be lowered to and represented as a while.
So, a template? I certainly ain't gonna be using go for its mustache support.
But as is all too common in the go community, there seems to be a lot of confusion about what is proposed, and resistance to any change.
The issue you linked was opened in 2019 and closed with no new comments in 2023, with active discussion through 2022.
Does Ruby strings already allow lazy processing ?
I'm not talking about wrapping them in a block and passing the block (all languages can do that with a lambdas) but a having literally that eventually resolves to something when you use it.
fmt.Sprintf("This house is %s tall", measurements(2.5))
fmt.Sprint("This house is ", measurements(2.5), " tall")
f"This house is {measurements(2.5)} tall"
Sprint is better in that regard, we can read from left to right without interruptions, but is a pain to write due to all the punctuation, nevermind refactor. For example, try adding a new variable between "This" and "house". With the f-string you just type {var} before "house" and you're done. With Sprint, you're now juggling quotation marks and commas. And that's just a simple addition of a new variable. Moving variables or substrings around is even worse.
Summing up, f-strings are substantially more ergonomic to use and since string formatting is so commonly done, this adds up quickly.
_log(f”My variable is {x + y}”)
_log(“My variable is {}”.format(x+y))
_log(“My variable is {z}”.format(z=x+y))
Even PEP 498 (fstrings) was a battle.
STR."Hello \{this.user.firstname()}, how are you?\nIt's \{tempC}°C today!"
compared to scala
s"Hello ${this.user.firstname()}, how are you?\nIt's ${tempC}°C today!"
STR."" ? really?
I am super excited this is finally accepted. I started working on PEP 501 4 years ago.
>>> template = 'Hello, {name}'
>>> template.format(name='Bob')
'Hello, Bob'
>>> template = f'Hello, {name}'
Traceback (most recent call last):
File "<python-input-5>", line 1, in <module>
template = f'Hello, {name}'
^^^^
NameError: name 'name' is not defined
I guess it’s more concise, but differentiating between eager and delayed execution with a single character makes the language less readable for people who are not as familiar with Python (especially latest update syntax etc).
EDIT: to flesh out with an example:
class Sanitised(str): # init function that sanitises or just use as a tag type that has an external sanitisation function.
def sqltemplate(name: Sanitised) -> str: return f”select * from {name}”
# Usage sqltemplate(name=sanitise(“some injection”))
# Attempt to pass unsanitised sqltemplate(name=“some injection”) # type check error
https://peps.python.org/pep-0750/#arbitrary-string-literal-p...
I mean, they took "yield" and @decorator, we have a trade deficit.
This looks really great! It's almost exactly like JavaScript tagged template literals, just with a fixed tag function of:
(strings, ...values) => {strings, values};
Take this type of nested template structure in JS:
html`<ul>${items.map((i) => html`<li>${i}</li>`}</ul>`
html(t"<ul>{map(lambda i: t"<li>{i}</li>", items)}</ul>")
In JavaScript reliable syntax highlighting and type-checking are keyed off the fact that a template can only ever have a single tag, so a static analyzer can know what the nested language is. In Python you could separate the template creation from the processing possibly introduce some ambiguities, but hopefully that's rare in practice.
I'm personally would be interested to see if a special html() processing instruction could both emit server-rendered HTML and say, lit-html JavaScript templates that could be used to update the DOM client-side with new data. That could lead to some very transparent fine-grained single page updates, from what looks like traditional server-only code.
Edit: Sorry I was snarky, its late here.
I already didn't like f-strings and t-strings just add complexity to the language to fix a problem introduced by f-strings.
We really don't need more syntax for string interpolation, in my opinion string.format is the optimal. I could even live with % just because the syntax has been around for so long.
I'd rather the language team focus on more substantive stuff.
Why stop there? Go full Perl (:
I think Python needs more quoting operators, too. Maybe qq{} qq() q// ...
[I say this as someone who actually likes Perl and chuckles from afar at such Python developments. May you get there one day!]
My issue with them is that you have to write your syntax in the string complex expressions dictionary access and such become awkward.
But, this whole thing is bike-shedding in my opinion, and I don't really care about the color of the bike shed.
My understanding of template strings is they are like f-strings but don't do the interpolation bit. The name binding is there but the values are not formatted into the string yet. So effectively this provides a "hook" into the stringification of the interpolated values, right?
If so, this seems like a very narrow feature to bake into the language... Personally, I haven't had issues with introducing some abstraction like functions or custom types to do custom interpolation.
The best use case I know of for these kinds of things is as a way to prevent sql injection. SQL injection is a really annoying attack because the "obvious" way to insert dynamic data into your queries is exactly the wrong way. With a template string you can present a nice API for your sql library where you just pass it "a string" but it can decompose that string into query and arguments for proper parameterization itself without the caller having to think about it.
That's exactly what it is. It's just that they use the word "council" instead of "committee".
Whether or not this is technically a swift call is in the eye of the beholder.
[0]: https://docs.python.org/3/library/string.html#template-strin...
def f(template: Template) -> str:
parts = []
for item in template:
match item:
case str() as s:
parts.append(s)
case Interpolation(value, _, conversion, format_spec):
value = convert(value, conversion)
value = format(value, format_spec)
parts.append(value)
return "".join(parts)
By contrast, here's the equivalent Ruby:
def f(template) = template.map { |item|
item.is_a?(Interpolation) ? item.value.convert(item.conversion).format(item.format_spec) : item
}.joinCan't think of a good reason now on why I would need this rather than just a simple f-string.
Any unsafe string input should normally be sanitized before being added in a template/concatenation, leaving the sanitization in the end doesn't seem like the best approach, but ok.
One of the PEP's developers, Lysandros, presented this in our local meetup, so I am passingly familiar with it, but still, I might be missing something.
I guess the crux of it is that I don't understand why it's `t"some string"` instead of `Template("some string")`. What do we gain by the shorthand?
Because it's new syntax, it allows for parsing the literal ahead of time and eagerly evaluating the substitutions. Code like
bar = 42
spam = t"foo {bar*bar} baz"
bar = 42
spam = Template("foo ", Interpolation(bar*bar), " baz")
With templates:
mysql.execute(t"DELETE FROM table WHERE id={id} AND param1={param1}")
mysql.execute("DELETE FROM table WHERE id=%s AND param1=%s", [id, param1])
But yeah it does seem a bit confusing, and maybe kinda not pythonic? Not sure.
[1] https://docs.python.org/3/library/string.html#template-strin...
edit: this was mentioned by milesrout in https://news.ycombinator.com/item?id=43649607
I recently asked him:
--
Hi David! I am a huge long time fan of SWIG and your numerous epic talks on Python.
I remember watching you give a kinda recent talk where you made the point that it’s a great idea to take advantage of the latest features in Python, instead of wasting your time trying to be backwards compatible.
I think you discussed how great f-strings were, which I was originally skeptical about, but you convinced me to change my mind.
I’ve googled around and can’t find that talk any more, so maybe I was confabulating, or it had a weird name, or maybe you’ve just given so many great talks I couldn’t find the needle in the haystack.
What made me want to re-watch and link my cow-orkers to your talk was the recent rolling out of PEP 701: Syntactic formalization of f-strings, which makes f-strings even better!
Oh by the way, do you have any SWIG SWAG? I’d totally proudly wear a SWIG t-shirt!
-Don
--
He replied:
Hi Don,
It was probably the "Fun of Reinvention".
https://www.youtube.com/watch?v=js_0wjzuMfc
If not, all other talks can be found at:
https://www.dabeaz.com/talks.html
As for swag, I got nothing. Sorry!
Cheers, Dave
--
Thank you!
This must be some corollary of rule 34:
https://www.swigwholesale.com/swig-swag
(Don’t worry, sfw!)
-Don
--
The f-strings section starts at 10:24 where he's live coding Python on a tombstone with a dead parrot. But the whole talk is well worth watching, like all his talks!
I’m having trouble understanding this - Can someone please help out with an example use case for this? It seems like before with an f string we had instant evaluation, now with a t string we control the evaluation, why would we further delay evaluation - Is it just to utilise running a function on a string first (i.e. save a foo = process(bar) line?)
You don't completely control the evaluation.
From the PEP:
> Template strings are evaluated eagerly from left to right, just like f-strings. This means that interpolations are evaluated immediately when the template string is processed, not deferred or wrapped in lambdas.
If one of the things you are interpolating is, as a silly example, an invocation of a slow recursive fibonacci function, the template string expression itself (resulting in a Template object) will take a long while to evaluate.
Are you saying that calling:
template = t”{fib_slow()}”
This is probably the best overview of why it was withdrawn:
https://mail.openjdk.org/pipermail/amber-spec-experts/2024-A...
That is, you must process a Template in some way to get a useful string out the other side. This is why Template.__str__() is spec'd to be the same as Template.__repr__().
If you want to render a Template like an f-string for some reason, the pep750 examples repo contains an implementation of an `f(template: Template) -> str` method: https://github.com/davepeck/pep750-examples/blob/main/pep/fs...
This could be revisited, for instance to add `Template.format()` in the future.
That said, I think this is a great bit of work and I look forward to getting to use it! Thank you!
sql"SELECT FROM ..."
re"\d\d[abc]"
There are also loads of people basically defaulting to "no" on new features, because they understand that there is a cost of supporting things. I will often disagree about the evaluation of that cost, but it's hard to say there is no cost.
Nobody wants a system that is unusable, slow, hard to implement for, or hard to understand. People sometimes just have different weights on each of these properties. And some people are in a very awkward position of overestimating costs due to overestimating implementation effort. So you end up in discussions like "this is hard to understand!" "No it isn't!"
Hard to move beyond, but the existence of these kinds of conversations serve, in a way, as proof that people aren't jumping on every new feature. Python is still a language that is conservative in what it adds.
This should actually inspire more confidence in people that features added to Python are _useful_, because there are many people who are defaulting to not adding new features. Recent additions to Python speeding up is more an indicator of the process improving and identifying the good stuff rather than a lowering of the bar.
[0]: I often think that these discussions often get fairly intense. Understandability is definitely a core Python value, but I Think sometimes discussions confuse "understandability" with "amount of things in the system". You don't have to fully understand pervasive hashing to understand Python's pervasive value equality semantics! A complex system is needed to support a simple one!
There have been processes put into place in recent years to try to curb the difficulty of things. One of those is that all new PEPs have to include a "how can you teach this to beginers" section, as seen here on this pep: https://peps.python.org/pep-0750/#how-to-teach-this
As Nick mentioned, PEP 750 had a long and winding road to its final acceptance; as the process wore on, and the complexities of the earliest cuts of the PEPs were reconsidered, the two converged.
[0] The very first announcement: https://discuss.python.org/t/pep-750-tag-strings-for-writing...
[1] Much later in the PEP process: https://discuss.python.org/t/pep750-template-strings-new-upd...
https://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
So, right now, you have two options to log:
1. `logger.debug(f'Processing {x}')` - looks great, but evaluates anyway, even if logging level > `logging.DEBUG`;
2. `logger.debug('Processing %s', x)` - won't evaluate till necessary.
What would be the approach with t-strings in this case? Would we get any benefits?
For a logger t-strings are mostly just a more pleasant and less bug-prone syntax for #2
Also: were prompt templates for LLM prompt chaining a use case that influenced the design in any way (examples being LangChain and dozens of other libraries with similar functionlity)?
The main reason for non having deferred evaluation was that it over-complicated the feature quite a bit and introduces a rune. Deferred evaluation also has the potential to dramatically increase complexity for beginners in the language, as it can be confusing to follow if you dont know what is going on. Which means "deferred by default" wasnt going to be accepted.
As for LLM's, it was not the main consideration, as the PEP process here started before LLM's were popular.
Maybe not directly, but the Python community is full of LLM users and so I think there's a general awareness of the issues.
The point of evaluation of the expressions is the same.
>>> template = t'Hello, {name}'
BUT the result of a t-string is not a string; it is a Template which has two attributes:
strings: ["Hello, ", ""]
interpolations: [name]
There is an observation that you can use `lambda` inside to delay evaluation of an interpolation, but I think this lambda captures any variables it uses from the context.
Actually lambda works fine here
>>> name = 'Sue'
>>> template = lambda name: f'Hello {name}'
>>> template('Bob')
'Hello Bob'Bummer. This could have been so useful:
statement_endpoint: Final = "/api/v2/accounts/{iban}/statement"
That's correct, they don't. Evaluation of t-string expressions is immediate, just like with f-strings.
Since we have the full generality of Python at our disposal, a typical solution is to simply wrap your t-string in a function or a lambda.
(An early version of the PEP had tools for deferred evaluation but these were dropped for being too complex, particularly for a first cut.)
def my_template(name: str) -> Template:
return t"Hello, {name}" >>> template = lambda name: f'Hello {name}'
>>> template('Bob')> just use a tag type and a sanitisation function that takes a string and returns the type
Okay, so you have a `sqlstring(somestring)` function, and the dev has to call it. But... what if they pass in an f-string?
`sqlstring(f'select from mytable where col = {value}')`
You havent actually prevented/enforced anything. With template strings, its turtles all the way down. You can enforce they pass in a template and you can safely escape anything that is a variable because its impossible to have a variable type (possible injection) in the template literal.
This example still works, the entire f-string is sanitised (including whatever the value of name was). Assuming sqlstring is the sanitisation function.
The “template” would be a separate function that returns an f-string bound from function arguments.
But normalizing one pattern ensures the whole community build API around it. This creates a unified ecosystem.
And it's a very clean API that is a no brainer for the string user.
modules, classes, protocols, functions returning functions, all options in Python, each work well for reuse, no need to use more than 2 at once, yet the world swims upstream.
evil = "<script>alert('evil')</script>"
sanitized = Sanitized(evil)
whoops = f"<p>{evil}</p>"If you create a subclass of str which has an init function that sanitises, then you can’t create a Sanitised type by casting right?
And even if you could, there is also nothing stopping you from using a different function to “html” that just returns the string without sanitising. They are on the same relative level of safety.
Agreed; it feels natural to accept plain templates (and simple sequences of plain templates) as HTML; this is hinted at in the PEP.
> html(t"<ul>{map(lambda i: t"<li>{i}</li>", items)}</ul>")
Perhaps more idiomatically: html(t"<ul>{(t"<li>{i}</li>" for i in items)}</ul>")
> syntax highlighting and type-checking are keyed off the fact that a template can only ever have a single tag
Yes, this is a key difference and something we agonized a bit over as the PEP came together. In the (very) long term, I'm hopeful that we see type annotations used to indicate the expected string content type. In the nearer term, I think a certain amount of "clever kludginess" will be necessary in tools like (say) black if they wish to provide specialized formatting for common types.
> a special html() processing instruction could both emit server-rendered HTML and say, lit-html JavaScript templates that could be used to update the DOM client-side with new data
I'd love to see this and it's exactly the sort of thing I'm hoping emerges from PEP 750 over time. Please do reach out if you'd like to talk it over!
html(['ul', {'class': 'foo'}, *(['li', item] for item in items)])
The benefit of lisp-like representation is you have the entire structure of the data, not just a sequence of already-serialized and not-yet-serialized pieces.
html(t"<ul>{(t"<li>{i}</li>" for i in items)}</ul>")One possibility would be to define __and__ on html so that you can write e.g. html&t"<b>{x}</b>" (or whichever operator looks the best).
def f(template: Template) -> str:
return "".join(
item if isinstance(item, str) else
format(convert(item.value, item.conversion), item.format_spec)
for item in template
)
def f(template):
return (for item in template:
isinstance(item, str) then item else
format(convert(item.value, item.conversion), item.format_spec)
).join('')Python has always been my preference, and a couple of my coworkers have always preferred Ruby. Different strokes for different folks.
Nah, idiomatic Python always used to prefer comprehensions over explicit loops. This is just the `match` statement making code 3x longer than it needs to be.
def _f_part(item) -> str:
match item:
case str() as s:
return s
case Interpolation(value, _, conversion, format_spec):
return format(convert(value, conversion), format_spec)
def f(template: Template) -> str:
return ''.join(map(_f_part, template))
I very frequently use this helper-function (or sometimes a generator) idiom in order to avoid building a temporary list to `.join` (or subject to other processing). It separates per-item processing from the overall algorithm, which suits my interpretation of the "functions should do one thing" maxim.
If I were tasked to modify the Python version to say, handle the case where `item` is an int, it would be immediately obvious to me that all I need to do is modify the `match` statement with `case int() as i:`, I don't even need to know Python to figure that out. On the other hand, modifying the Ruby version seems to require intimate knowledge of its syntax.
I don't particularly love the Ruby code either, though - I think the ideal implementation would be something like:
fn stringify(item) =>
item.is_a(Interpolation) then
item.value.convert(item.conversion).format(item.format_spec)
else item.to_string()
fn f(template) => template.map(stringify).join()
What do you mean? Python has always been that way. "Explicit is better than implicit. [..] Readability counts." from the Zen of python.
> By contrast, here's the equivalent Ruby:
Which is awful to read. And of course you could write it similar short in python. But it is not the purpose of a documentation to write short, cryptic code.
Almost all Python programmers should be familiar with list comprehensions - this should be easy to understand:
parts = [... if isinstance(item, Interpolation) else ... for item in template]
parts = []
for item in template:
match item:
case str() as s:
parts.append(...)
case Interpolation(value, _, conversion, format_spec):
parts.append(...)
I think for someone with a basic knowledge of Ruby, it's more understandable than the Python. It's a combination of basic Ruby features, nothing advanced.
I don't particularly love Ruby's syntax either, though - I think the ideal implementation would be something like:
fn stringify(item) =>
item.is_a(Interpolation) then
item.value.convert(item.conversion).format(item.format_spec)
else item.to_string()
fn f(template) => template.map(stringify).join() fn stringify(item) =>
item.is_a(String) then item else
item.value.convert(item.conversion).format(item.format_spec)
fn f(template) => template.map(stringify).join() > Not so bad for one argument but gets linearly worse.
<?php ... ?><some_markup>...<? php ... ?><some_more_markup here>...
https://discuss.python.org/t/pep-750-tag-strings-for-writing...
> Backticks are traditionally banned from use in future language features, due to the small symbol. No reader should need to distinguish ` from ' at a glance. It’s entirely possible that the prevailing opinion on this has changed, but it’s certainly going to be easier to stick to the letter prefixes and regular quotes.
and earlier in the thread the difficulty in typing it for some is cited as another reason
What t-strings offer over f-strings is the ability to control how the final string is put together from the calculated results. And the t-string doesn't have `__str__` - you have to explicitly pass it to a named formatting function to get a string. (There is `__repr__, but that's for debugging, of course.) So you can potentially reuse the same Template instance (created from the t-string) multiple times in different contexts to format the same information different ways.
I'd agree that it should be somewhere within the library, even if it's just a separate top-level function in the described `string.templatelib`. If it isn't, I'm sure someone will make a PyPI package for it right away.
any_func(f"{attacker_provided}") <=> eval(attacker_provided), from a security/correctness perspective
Ruby has lazy evaluation with a generic lazy enumeration facility, whether to produce string or any kind of object.
That is, I don't know what is the actual behavior of the default string interpolation in Ruby, but if profiling a codebase some string generation would gain lazy evaluation, there is a path to do so. But in the general case, does it really matter? Chances are good that a string construction is not a big bottleneck.
Does Python miss such a feature of generic lazy enumeration, or is it so painful to use that some syntactic sugar felt like a must have? Genuine question here, I don't have any strong opinion on this t-string feature.
- string construction is a hot path, you don't want them to always be lazy, especially since any access is slow in python.
- having it using a string syntax is just very clean and easy to read. It's explicit and can be supported by good editor highlighting.
- it's easy to grep, analyse for, substitute, etc.
- you get one single unified API instead of thousands of variations. Translations API, log API and escaping API can all look the same, arguments are in the same shapes.I understand that string generation can be a hotpath, though I wouldn't take it as a general certain fact.
From what I understand here the benefit in term of performance is mainly due to partial application automatically handled by the interpreter. It's hard to me to jauge actual pro/con compared to Ruby which can also leverage on freezed string, lambda, miscellaneous lazy evaluation facilities for example. I'm not aware of anything close in PHP, to stay in the realm of popular interpreted languages. I didn't make any Lua for a long time, so no idea how it evolved on that matter.
`some template {someVar}` was f-strings and someFunction`some template {someVar}` was more like what these t-strings provide to Python. t-strings return an object (called Template) with the template and the things that go into the "holes", versus tagged templates are a function calling pattern, but t-strings are still basically the other, richer half of ES2015+ template strings.
I do this myself. I basically always use the subtl wrong log.warning(f"Unexpected {response=} encountered") and not the correct, and depending on the loglevel cheaper log.warning("Unexpected respone=%s encountered", repsonse). The extra visual noise is typically not worth the extra correctness and performance (I'd obviously not do this in some publically exposed service receiving untrusted inputs).
I'd argue these use cases are in fact more prevalent then the bashing unstructured text use case.
Encouraging people to write injection vulnerabilities or performance and correcness bugs isn't great language design.
On the other hand, there’s a difference in localizability as well: the latter is localizable, the former isn’t. (It also worries me that I see no substantive discussion of localization in PEP 750.)
First—and more importantly—a professional translator’s interface is list of strings and some supplementary materials in, list of strings out. (Protip: given baseline i18n competence like not concatenating sencences out of parts, the quality of the translation you get is largely determined by the supplementary materials; screenshot every part of your UI and your translator will be willing to kiss you.) Both in communicating with clients and on the fast path of their own work (in CAT software like Across, Trados, etc.).
They do not need to see the code that fills in any placeholders, nor do they want to spend time and attention preserving it exactly as it’s been written—they just want to reorder the placeholders as the syntax of the language dictates. Arguably even %s vs %d is too much information. The ideal is {name} and {count}, and {1} and {2} are acceptable.
(By contrast, a very common issue is that like half of the sentence may need to change depending on the numeric value filled in for {count}, and two variants in English may map to anywhere between one and four after localization[1]. Template strings help with this not at all.)
Second—though this may be easier to fix—the two major ways to retrieve localized messages at runtime are integers in, strings out (most commercial tools out there) and strings in, strings out (GNU gettext), where the output strings are retrieved from some sort of data file that’s intentionally incapable of containing executable code (resource-only “MUI” DLLs, hashtables in GNU “MO”s, plain old text in Java “properties”, various kinds of XML, etc.).
Operating on pure data isn’t a strict necessity. It’s is largely a concession to the development process, which may not permit the string tables to go out to the localization contractors until principal QA is finished. The last thing you want is to insert engineering into the already-slow loop between the translators, the editors (or a translation agency employing both), and localization QA (hopefully in-house, using the actual live software to check the translations in context).
[1] https://www.gnu.org/software/gettext/manual/html_node/Plural...
Then there's "You can [get] a similar effect using fmt.Sprint, with custom functions for non-default formatting." [2]:
- Just the fact that "you can already do this" needs to be said should give the designers pause. Clearly you can't already do this if people are requesting a new feature. Indeed, this situation exactly mimics the story of Go's generics - after all, they do not let you do anything you couldn't do before, and yet they got added to Go. It's as if ergonomics matter, huh.
Another way to look at this: if fmt.Sprint is so good it should be used way more than fmt.Sprintf right? Should be easy to prove :)
- The argument crumbles under the load-bearing "similar effect". I already scratched the surface of why this is wrong in a sibling post: [3].
I suspect the reason for this shallow dismissal is the designers didn't go as far as to A/B test their proposal themselves, so their arguments are based on their gut feel instead of experience. That's the only way I can see someone would come up with the idea that fmt.Sprint and f-strings are similar enough. They actually are if all you do is imagine yourself writing the simplest case possible:
fmt.Sprint("This house is ", measurements(2.5), " tall")
f"This house is {measurements(2.5)} tall"
P.S. Proposing syntax as noisy as:
fmt.Println("This house is \(measurements(2.5)) tall")
[1]: https://github.com/golang/go/issues/57616#issuecomment-14509...
[2]: https://github.com/golang/go/issues/34174#issuecomment-14509...
Are you objecting to the use of `\(…)` here instead of `{…}`? Because of the extra character or because of the need to nest parentheses?
Being familiar doesn't mean it's readable. They can be useful, but readability is usually not on that list.
> I think for someone with a basic knowledge of Ruby, it's more understandable than the Python.
I know both, and still consider it awful. Readability is not about making it short or being able to decipher it.
The t-string proposal involves using new data types to abstract the concatenation and formatting process, but it's still a compile-time process - and the parts between the braces still involve code that executes first - and there's still no separate type for the overall t-string literal, and no way to end up eval'ing code from user-supplied data except by explicitly requesting to do so.
Python source code is translated into bytecode for a VM just like in Java or C#, and by default it's cached in .pyc files. It's only different in that you can ask to execute a source code file and the compilation happens automatically before the bytecode-interpretation.
`SyntaxError` is fundamentally different from other exceptions because it can occur during compilation, and only occurs at run-time if explicitly raised (or via explicit invocation of another code compilation, such as with `exec`/`eval`, or importing a module). This is also why you can't catch a `SyntaxError` caused by the invalid syntax of your own code, but only from such an explicit `raise` or a request to compile a source code string (see https://stackoverflow.com/questions/1856408 ).
It is NOT about the possibility of referencing existing / future (lazy / deferred evaluation) string literals from within the string, but about a format string that would literally evaluate arbitrary functions within a string.
On the other hand, the current solution offered by Go (fmt.Sprintf) is the one who supports a user-supplied format String. Admittedly, there is a limited amount of damage that could be done this well, but you can at the very least cause a program to panic.
The reason for declining this feature[1] has nothing to do with what you stated. Ian Lance Taylor simply said: "This doesn't seem to have a big advantage over calling fmt.Sprintf" and "You can a similar effect using fmt.Sprint". He conceded that there are performance advantages to string interpolation, but he doesn't believe there are any gains in usability over fmt.Sprintf/fmt.Sprint and as is usual with Go (compared to other languages), they're loathe to add new features to the compiler[2].
[1] https://github.com/golang/go/issues/34174#issuecomment-14509...
[2] https://github.com/golang/go/issues/34174#issuecomment-53013...
> they are better for most use cases of string templating where what you really want, is just a string.
I think use cases where you want to unconditionally bash a string together are rare. I'd bet that in > 80% of cases the "just a string" really is just a terrible representation for what really is either some tree (html, sql, python, ...) structure or at least requires lazy processing (logging, where you only want to pay for the expensive string formatting and generation if you run at the log level or higher that the relevant logging line is meant to operate).
"just a string" where the string really is just a string is really, really common though. Python is the go-to language for a lot of people (including me) if they just want to bash together a one-time-use script that will never see input from other people, and f-strings are incredibly useful in such cases for generating filenames, debugging messages, etc.
Kinda messy PEP, IMO, I'm less excited by it than I'd like to be. The goal is clear, but the whole design feels backwards.
The Rejected Golang proposal cited by the post I'm replying to. NOT Python's present PEP or any other string that might resolve magic variables (just not literally eval / exec functions!).
However the difficulty of understanding also illustrates the increased maintenance burden and language complexity.
The proposal was for the same.
You can verify that either via static typechecking, or at runtime.
There is no such function; Template.__str__() returns Template.__repr__() which is very unlikely to be useful. You pretty much have to process your Template instance in some way before converting to a string.
They are both similar in their unsafety.
There are a lot of existing workarounds in the discussions if you are interested enough in using it, such as using lambdas and t-strings together.
statement_endpoint: Final = "/api/v2/accounts/{iban}/statement".format
The ideal version has the same behaviour and shows that the extra complexity is unnecessary.
Yes, you can do that, but a t-string is much more ergonomic, and IMO, more readable .
Other than a more broad "how is the language as a whole faring?" test, which might be done through surveys or other product-style research, I think this is just plainly a hard problem to approach, just by the nature that it's largely about user experience.
"Some hammers are just shaped weird, oh well, just make do with it."
For example, some people that I interview does not "get" why you have to initialize the dict before doing dict[k] += 1. They know that they have to do some ritual of checking for k in dict and dict[k] = 0. But they don't get that += desugars into dict[k] = dict[k] + 1.
You cant throw an error on unsanitized because the language has no way to know if its sanitized or not. Either way, its just a string. "returning an f-string" is equivalent to returning a normal string at runtime.
> it has no way of knowing if it’s sanitised or not
It does. You define the SanitisedString class. Constructing one sanitises the string. Then when you specify that as the argument, it forces the user to sanitise the string.
If you want to do it without types, you can check with `isinstance` at runtime, but that is not as safe.
Comprehensions, though -- they are perfection. :-)
That would make the mapping between a comprehension and the equivalent loop much clearer, especially once you use nested loops and/or conditionals.
For example, to flatten a list of lists `l = [[1, 2], [3], [4, 5, 6]]`:
[item for sublist in l for item in sublist]
[for sublist in l: for item in sublist: item]I think I'm following more, and I see how you can accomplish this by encapsulating the rendering, but I'm still not seeing how this is possible with user facing f-strings. Think you can write up a quick example?
So the thing I'm still not getting from your example is allowing the template itself to be customized.
evil = "<script>alert('evil')</script>"
template1 = t"<p>{evil}</p>"
template2 = t"<h1>{evil}</h1>"
html(template1)
html(template2)As for the logic, I would still use pattern matching for branching and destructuring, but I’d put it in a helper. More lines is not a negative in my book, though I admit the thing with convert and format is weird.
Yeah, using a helper function makes things much clearer. To be honest though, I'm not a huge fan of using either `isinstance` (which is generally a sign of a bad design) nor `match/case` (which is essentially a "modern" way to write `isinstance`).
I can't help but think that a better design could avoid the need for either of those (e.g. via polymorphism).
Haskell also has operator overloading on steroids so you could use the (|>) operator from Flow and write transformations the same as you would shell pipes. I’d love to whip up an example but it’s difficult on this tiny screen. Will try to remember when I’m on my computer.
Before someone chimes in with ML propaganda, I warn you that I’m going to exercise my rights under the Castle Doctorine the moment you say “of”.
I wrote it up (https://news.ycombinator.com/item?id=43650001) before reading your comment :)
E.G. Adapting https://github.com/golang/go/issues/34174
f := 123.45
fmt.Fprintln("value=%08.3f{f}") // value=0123.450
fmt.Fprintln("value=%08.3f", f) // value=0123.450
s := "value"
fmt.Fprintln("value='%50s{s}'") // value='<45 spaces>value'
fmt.Fprintln("value='%50s'", s) // value='<45 spaces>value'
fmt.Fprintln("value=%08.3f%{f}") // value=0123.450
fmt.Fprintln("value='%50s%{s}'") // value='<45 spaces>value'Using the SanitisedString type forces the user to explicitly call a sanitiser function that returns a SanitisedString and prevents them from passing in an unsanitised str.
With t-strings the rendering function is responsible for sanitization, and users can pass unrendered templates to it.
With f-strings there's no concept of an unrendered template, it just immediately becomes a string. Whoever is creating the template therefore has to be careful what they put in it.
t-strings allow a library to perform transformations on the values, such as escaping them, or passing them as separate values to a parameterized query. Escaping html and parameterizing sql queries were the first two example use cases given in the PEP.
And I disagree that such use cases are niche. In my experience, needing to sanitize user input is an extremely common thing to need to do, and having the language and library make it as easy as possible to do it correctly is a very good thing.
[1]: I do wish they hadn't called these Templates, because it's not really a template so much as an intermediate representation of an interpolated value.
Pre-compilation means that you first compile the template, then you supply the template with values when you render it multiple times. This is not possible with t-strings since the values are bound when the t-string is created.
With respect to compilation, that is basically is how t-strings work, but it is the python interpreter that does the compilation. When it parses the t-string, it compiles it to (byte) code to generate a Template object from from the expressions in scope when it is evaluated, which may happen more than once. And if you really want a template that is a separate object that is passed the values separately, you can just wrap a t-string in a function that takes the parameters as arguments.
> two dozen templating libraries that offer much more comprehensive safe and fast text-generation solutions than what t-strings do
But t-strings allow those libraries to be safer (users are less likely to accidentally interpolate values in an f-string, if a t-string is required) and possibly faster (since the python interpreter does the hard work of splitting up the string for you. t-strings don't replace those libraries, it allows them to be better.
> And if you really want a template that is a separate object that is passed the values separately, you can just wrap a t-string in a function that takes the parameters as arguments.
No, you can't do that: "Template strings are evaluated eagerly from left to right, just like f-strings. This means that interpolations are evaluated immediately when the template string is processed, not deferred or wrapped in lambdas." Every function evaluation creates a new Template object, it does not reuse a precompiled one.
> and possibly faster
Possibly not, since precompilation is not supported.
My main motivation as an author of 501 was to ensure user input is properly escaped when inserting into sql, which you cant enforce with f-strings.
I used to wish for that and got it in JS with template strings and libs around it. For what it’s worth (you got a whole PEP done, you have more credibility than I do) I ended up changing my mind, I think it’s a mistake.
It’s _nice_ from a syntax perspective. But it obscures the reality of sql query/parameter segregation, it builds an abstraction on top of sql that’s leaky and doesn’t even look like an abstraction.
And more importantly, it looks _way too close_ to the wrong thing. If the difference between the safe way to do sql and the unsafe way is one character and a non-trivial understanding of string formatting in python… bad things will happen. In a one-person project it’s manageable, in a bigger one where people have different experiences and seniority it will go wrong.
It’s certainly cute. I don’t thing it’s a good thing for sql queries.
One statement the PEP could put front and center in the abstract could be "t-strings are not strings".
Example, if you can go through (I'm not sure you can) and trivially replace all your fs with ts, and then have some minor fixups where the final product is used, I don't think a migration from one to the other would be terribly painful. Time-consuming, yes.
https://peps.python.org/pep-0750/#no-template-str-implementa...
I don't know what you mean by `executable` streams, but besides databases as I've already mentioned, a common thing that shows up in non-web applications is invoking a shell command that includes a user-supplied file name as part of it. Currently doing so safely means you need to call `shlex.quote` or similar on the filename, but with t-strings you could have something like: `shell(t"some-command {filename} 2> somefile | other-command")`.
And that is just one specific example. There are other cases it might be useful as well, like say generating an XML configuration file from a template that includes user-supplied input.
> No, you can't do that... Every function evaluation creates a new Template object, it does not reuse a precompiled one.
The code that generates that Template object is pre-compiled though.
If you define a function like:
def my_template(a, b,c):
return t"a={a} b={b} c={c}"
def my_template(a, b,c):
return Template("a=", Interpolation(a, ...), " b=", Interpolation(b, ...), " c=", Interpolation(c,...))
t-string is an unfortunate name for something that is not a string.
"foo %s" % "bar"
"foo {}".format("bar")
bar = "bar"; f"foo {bar}"
bar = "bar"; t"foo {bar}" # has extra functionality!It does suck for beginners who end up having to know about all variations until their usage drops off.
The other ones at least are based on the same format string syntax.
"foo {}".format("bar") would be an obvious "just use f-string" case, except when the formatting happens far off. But in that case you could "just" use t-strings? Except in cases where you're (for example) reading a format string from a file. Remember, t- and f- strings are syntactic elements, so dynamism prevents usage of it!
So you have the following use cases:
- printf-style formatting: some C-style string formatting is needed
- .format: You can't use an f- string because of non-locality in data to format, and you can't use a t- string due to dynamism in
- f-string: you have the template and the data in the same spot lexicographically, and you just want string concatenation (very common!)
- t-string: you have the template and the data in the same spot lexicogrpahically, but want to use special logic to actually build up your resulting value (which might not even be a string!)
The last two additions being syntax makes it hard to use them to cover all use cases of the first two.
But in a specific use case? It's very likely that there is an exact best answer amongst these 4.
It’s also the only one which is anything near safe for being user provided.
I see (4) being about the flexibility of (2) and readability of (3). Maybe it'll eventually grow to dominate one or both, but it's also fine if it doesn't. I don't see (1) going away at all since the curly collision still exists in (4).
import string
t = string.Template("foo $bar")
t.substitute(bar="bar")log.error("foo happend %s", reason)
printf-style... does not support any of that. It can only format the objects passed in.