Building an early warning system for LLM-aided biological threat creation
Building an early warning system for LLM-aided biological threat creation
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat. In an evaluation involving both biology experts and students, we found that GPT-4 provides at most a mild uplift in biological threat creation accuracy. Whil...
OpenAI blog post: https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation
Orange discuss: https://news.ycombinator.com/item?id=39207291
I don't have any particular section to call out. May post thoughts tomorrow today it's after midnight oh gosh, but wanted to post since I knew ya'll'd be interested in this.
Terrorists could use autocorrect according to OpenAI! Discuss!
You're viewing a single thread.
from the orange site thread:
Neural networks are not new, and they're just mathematical systems. LLMs don't think. At all. They're basically glorified autocorrect. What they're good for is generating a lot of natural-sounding text that fools people into thinking there's more going on than there really is.
Obvious question: can Prolog do reasoning?
If your definition of reasoning excludes Prolog, then... I'm not sure what to say!
this is a very specific sneer, but it’s a fucking head trip when you’ve got in-depth knowledge of whichever obscure shit the orange site’s fetishizing at the moment. I like Prolog a lot, and I know it pretty well. it’s intentionally very far from a generalized reasoning engine. in fact, the core inference algorithm and declarative subset of Prolog (aka Datalog) is equivalent to tuple relational calculus; that is, it’s no more expressive than a boring SQL database or an ECS game engine. Prolog itself doesn’t even have the solving power of something like a proof assistant (much less doing anything like thinking); it’s much closer to a dependent type system (which is why a few compilers implement Datalog solvers for type checking).
in short, it’s fucking wild to see the same breathless shit from the 80s AI boom about Prolog somehow being an AI language with a bunch of emphasis on the AI, as if it were a fucking thinking program (instead of a cozy language that elegantly combines elements of a database with a simple but useful logic solver) revived and thoughtlessly applied simultaneously to both Prolog and GPT, without any pause to maybe think about how fucking stupid that is
Obvious question: can Prolog do reasoning? If your definition of reasoning excludes Prolog, then… I’m not sure what to say!
Oh, I don't know, maybe that reasonable notions of "reasoning" can include things other than mechanistic search through a rigidly defined type system. If Prolog is capable of reasoning in some significant sense that's not fairly reasonably achieved with other programming languages, how come we didn't have AGI in the 70s (or indeed, now)?
You're not alone. I like Prolog and I feel your pain.
That said I think Prolog can be a particularly insidious Turing tarpit, where everything is possible but most things that feel like a good match for it are surprisingly hard.
That said I think Prolog can be a particularly insidious Turing tarpit, where everything is possible but most things that feel like a good match for it are surprisingly hard.
oh absolutely! I’ve been wanting to go for broke and do something ridiculous in Prolog like a game engine (for a genre that isn’t interactive fiction, which Prolog excels at if you don’t mind reimplementing big parts of what Inform provides) or something that touches hardware directly, but usually I run into something that makes the project unfun and stop.
generally I suspect Prolog might be at its best in situations where you really need a flexible declarative language. I feel like Prolog might be a good base for a system service manager or an HDL. but that’s kind of the tarpit nature of Prolog — the obvious fun bits mask the parts that really suck to write (can I even do reliable process management in Prolog without a semi-custom interpreter? do I even want to juggle bits in Prolog at all?)
one of the most recent things I've seen in this space is https://www.biscuitsec.org/, which is built on datalog and aims to solve a problem in a fairly interesting domain. I still mean to try it out on a few things, to see how well it maps to use in reality
that seems very cool! I’ve been frustrated in the past by rules-based auth libraries implementing half-baked but complex declarative DSLs when Datalog is right there, so I’m hoping it works well in practice because I’d love to use it too
what, you don't like the 10~15y old pattern of someone slapping together a DSL in a weekend because they read a blogpost about it last week, and then having to deal with the evolving half-restricted half-allows-eval mess in [ruby,erlang,...] with its syntax denoted in some way that isn't equivalent between operating languages? sheesh. what kind of modern web engineer are you?!
lukewarm take: the fact that "yaml engineer" exists as a joking self-deprecating referential description of what so many people do is both an indictment of their competencies (so, so many of these people would rather twiddle variables than even think of learning to write a small bit of programming), but also of the tools that claim to provide more abstractions and an "easier way" to do things
(yes I have a whole rant about this bullshit stored up)
one day the things we do with yaml will correctly be seen as a crime, but very likely only after yaml is replaced by something significantly worse, cause our field stubbornly refuses to learn a damn thing. it’s probably not a coincidence that the only declarative languages I know that aren’t monstrosities are from academia, and they’re extremely unpopular compared to the approach where a terrible heap of unreadable yaml is made worse by shoving an awful macro language into every field
""" just as They have erased the pyramid building knowledge from our historic memory, They just don't want you to know that Prolog really solved all of this in the 80s. Google and OpenAI are just shitty copies - look how wasteful their approaches are! all of this javascript, and yet... barely a reasoned output among it all
told you kid, the AI Winter never stopped. don't buy into the hype """
[Datalog] is equivalent to tuple relational calculus
Well, Prolog also allows recursion, and is Turing complete, so it's not as rudimentary as you make it out to be.
But to anyone even passingly familiar with theoretical CS this is nonsense. Prolog is not "reasoning" in any deeper sense than C is "reasoning", or that your pocket calculator is "reasoning". It's reductive to the point of absurdity, if your definition of "reason" includes Prolog then the Brainfuck compiler is AGI.
Datalog is specifically a non-TC subset of Prolog with a modified evaluation strategy that guarantees queries always terminate, though I was being imprecise — it’s the non-recursive subset of Datalog that’s directly equivalent to TRC (though Wikipedia shows this by mapping Datalog to relational algebra, whereas I’d argue the mapping between TRC and Datalog is even easier to demonstrate). hopefully my imprecision didn’t muddy my point — the special sauce at Prolog’s core that folks seem to fetishize is essentially ordinary database shit, and the idea of a relational database having any kind of general reasoning is plainly ridiculous.