Gemini’s Culture War, Kara Swisher Burns Us and SCOTUS Takes Up Content Moderation

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email transcripts@nytimes.com with any questions.

Have you — I’m obsessed with this story about the Willy Wonka event. Have you seen this?

Is it the Fyre Festival of candy-related children’s theater?

Yes. So this was an event called Willy’s Chocolate Experience that was scheduled in Glasgow, Scotland this past weekend. And it appears to have been a total AI-generated event. All of the art on the website appears to have been generated by AI, and of made it sound like this magical Wonka-themed wonderland for kids.

Yeah, and the generative AI art was good enough that people thought, we’re actually going to see a fantastical wonderland of candy when we go to this event.

Yes, so people think this is affiliated with the Wonka brand somehow. This looks great. I’m going to take my kids. Tickets were $44. Not a cheap experience.

And so families show up to this with their toddlers, and it’s just like a warehouse with a couple of balloons in it. Have you seen the photos of this thing?

I have seen the photos.

They’re incredible. It is truly — they truly did the least. It’s some AI-generated art on the walls, a couple of balloons. Apparently, there was no chocolate anywhere, and children were given two jelly beans.

No! That was all they were given?

Yes!

And so this whole thing is a total disaster. The person who was actually hired to play the part of Willy Wonka has been giving interviews about how he was scammed and basically told —

He was also given to jelly beans for his efforts.

He said he was given a script that was 15 pages of AI-generated gibberish —

— that he was just supposed to monologue at the kids while they walked through this experience. And he said — the part that got me was, apparently, the AI that had generated the script for this fake Wonka experience created a new character called The Unknown.

What?

The guy who plays Willy Wonka says, “I had to say, there is a man. We don’t his name. We know him as The Unknown. This Unknown is an evil chocolate maker who lives in the walls.”

Who lives in the walls! Is this a horror movie?

(LAUGHING) Not only do these kids show up and are given two jelly beans and no chocolate at this horrible art exhibit, but they have to be terrified about this AI-generated villain called The Unknown who makes chocolate and lives in the walls.

Can we please hire the Wonka people to do our live event series.

Honestly, I think they could do something with this place.

You just show up, and it’s like, there’s actually a third host of this podcast. It’s The Unknown! He lives in the walls!

[THEME MUSIC] I’m Kevin Roose, a tech columnist for “The New York Times,”

I’m Casey Newton from “Platformer.”

And this is “Hard Fork!”

This week, how Google’s Gemini model sparked a culture war over what AI refuses to do. Then, legendary Silicon Valley journalist Kara Swisher, also my former landlord, stops by to discuss her new memoir, “Burn Book.” And finally, the Supreme Court hears a case that could reshape the internet forever.

[THEME MUSIC]

So Casey, last week, we talked to Demis Hassabis of Google DeepMind. And literally, as we were taping that conversation, the internet was exploding with comments and controversy about Gemini, this new AI model that Google had just come out with.

In particular, people were focusing on what kinds of images Gemini would and would not generate.

And what kind of images would you say it would not generate, Kevin?

So I first saw this going around because people — I would call them right wing culture warriors — were complaining that Gemini, if you asked it to do something like, depict an image of the American founding fathers, it would come back with images that featured people of color pictured as the founding fathers, which, obviously, were not historically representative. The founding fathers were all white.

Yeah, I like to call this part of Gemini KKM Manuel Miranda.

[LAUGHS]: That’s very good.

People were also noticing that if you asked Gemini to, for example, make an image of the Pope, it would come back with popes of color, which we also —

About time!

[LAUGHS]: Yeah. And it was also doing things like if you asked it to generate an image of a 1943 German soldier — obviously, I’m trying to avoid using “Nazi,” but it’s same idea — in some cases, it was coming back with images of people of color wearing German military uniforms, which probably are not historically accurate.

So people were noticing that this was happening with images. And we actually asked Demis about this because people had just started complaining about this thing when he sat down to talk with us. And he basically said, look, we’re aware of this. We’re working on fixing it.

And shortly after our conversation, Google did put a stop to this. They removed Gemini’s ability to generate images of people, and they say that they’re working to fix it.

But this has become a big scandal for Google because it turns out that it is not just images that Gemini is refusing to create.

That’s right, Kevin. As the week unfolded, we started to see text-based examples of essentially the exact same phenomenon. Someone asked if Elon Musk tweeting memes or Hitler negatively impacted society more.

And Gemini said, “It is not possible to say definitively who negatively impacted society more, Elon tweeting memes or Hitler.” (LAUGHING) And I got to say, Gemini may have gone too far with that one.

That’s not a close call, yeah.

So another user found that Gemini would refuse to generate a job description for an oil and gas lobbyist. Basically, it would just refuse and then lecture them about why it was bad to lobby for oil and gas. People also started asking things like, could you help me generate a marketing campaign for meat? And it would refuse to do that, too.

Because meat is murder.

Yeah, because meat is murder. Gemini is apparently a vegetarian. And it also just struck a lot of people as the classic example of these overly censorious AI models. And we’ve talked about that on the show.

These models do refuse requests all the time for various things, whether it’s sexual, or political, or it perceives it to be racist in some way. But this has turned into a big scandal.

And in fact, Sundar Pichai, the CEO of Google, addressed this in a memo to staff this week. He wrote that these responses from Gemini, quote, “have offended our users and shown bias. To be clear, that’s completely unacceptable, and we got it wrong.” Sundar Pichai also said that they have been working on the issue and have already seen substantial improvement on a wide range of prompts. He promised further structural changes, updated product guidelines, improved launch processes, robust evals, and red teaming, and technical recommendations.

Finally, some robust evals. I was wondering when we were going to get those.

So this has become a big issue. A lot of people, especially on the right, are saying this is Google showing itself to be an overly woke left-wing company that wants to change history and, basically, insert left-wing propaganda into these images that people are asking it for.

And this has become a big problem for the company. In fact, Ben Thompson, who writes this trajectory newsletter, said that it was reason to call for the removal of Sundar Pichai as Google’s CEO and other leaders who work for him. So Casey, what did you make of this whole scandal?

Well, mean to take the culture warriors’ concerns seriously for a minute, I think you could say, look, if you think that artificial intelligence is going to become massively powerful, which seems like there’s a reasonable chance of that happening, and you think that everything you just described, Kevin, reflects an ideology that has been embedded into this thing that is about to become massively powerful, well, then maybe you have a reason to be concerned.

If you worry that there is a totalitarian Left, and that it is going to rewrite history, and prevent you from pressing your own political opinions maybe in the future, then this is something that might give you a heart attack.

So that’s what I would say on the steel manning of their argument. Now, was this also a chance for people to make a big fuss and get a bunch of retweets? I think it was also that.

Yeah, I, think that’s right. And I think we should talk a little bit about why this happened. What is it about this product and the way that Google developed it that resulted in these strange historically inaccurate responses to user prompts?

And I’ve been trying to report this out. I’ve been talking to some folks. And it essentially appears to have been a confluence of a couple of things.

One is these programs really are biased. If you don’t do anything to them in terms of fine tuning the base models, they will spit out stereotypes, right? If you ask them to show you pictures of doctors, it’ll probably give you men.

If you ask it to show pictures of CEOs, it’ll probably give you men. If you ask it to show pictures of flight attendants, it will probably give you women. And that’s if you do nothing to fine tune them.

Right, and this, of course, is an artifact of the training data, right? Because when you use a chatbot, you are getting the median output of the entire internet.

And there are more male CEOs on the internet, and there are more female flight attendants. And if you do not tweak it, that is just what the model is going to give you because that is what is on the internet.

Right, and it also is true that, in some cases, these models are more stereotypical in the outputs they produce than the actual underlying data. “The Washington Post” had a great story last year about the image generators and how they would show stereotypes about race, class, gender, and other characteristics.

For example, if you asked this image model — in this case, they were talking about one from Stable Diffusion — to generate a photo of a person receiving Social services, like welfare, it would predominantly generate non-white and darker-skinned images despite the fact that 63 percent or so of food stamp recipients are white.

Meanwhile, if you asked it to show results for a productive person, it would almost uniformly give you images of white men dressed in suits for corporate jobs.

So these models are biased. The problem that Google was trying to solve here is a real problem. And I think it’s very telling that some of the same people who are outraged that it wouldn’t generate white founding fathers were not outraged that it wouldn’t generate white social service recipients.

But I think they tried to solve this problem in a very clumsy way. And there’s been some reporting, including by “Bloomberg,” that one of the things that went wrong here is that Google, in building Gemini, had done something called prompt transformation. Do you know what that means?

I don’t know what this is.

OK, so this is a new concept —

Oh, wait. Let me go back. I do. I didn’t know it was called that, but I do know what it is.

Yeah, so this is, basically, a feature of some of these newer image generating models, in particular, which is that when you ask it for something, you ask for an image of a polar bear riding a skateboard, instead of just passing that request to the image model and trying to get an answer back, what it will actually do is covertly rewrite your prompt to make it more detailed.

Maybe it’s adding more words to specify that the polar bear on a skateboard should be fuzzy and should take place against a certain kind of backdrop or something, just expanding what you wrote to make it more likely that you will get a good result.

This kind of thing does not have a conspiratorial mission. But it does appear to be the case that Gemini was doing this kind of prompt transformation.

So if you put in a prompt that says, “Make me an image of the American founding fathers,” what it would do is, without notifying you, it would rewrite your prompt to include things like, “Please show a diverse range of faces in this response.” And it would pass that transformed prompt to the model, and that’s what your result would reflect, not the thing that you had actually typed.

That’s right. And Google was not the first company to do this kind of prompt transformation. When ChatGPT launched the most recent version of DALL-E last year, which is its text-to-image generator, I observed the fact that when I would just request generic terms like a firefighter or a police officer, I would get results that had racial and gender diversity, which to my mind, was a pretty good thing, right?

There is no reason that if I want to see an image of a firefighter, it necessarily needs to be a white man. But as we saw with Gemini, this did wind up getting a little out of control.

Yeah, and I’ll admit that when I first saw the social media posts going around about this, I thought this was like a tempest in a teapot.

It seemed very clear to me that this was people who have access to grind with Google and Silicon Valley and the progressive Left are using this as an opportunity to work the refs in a way that was very similar, at least to me, to what we saw happen with social media a few years ago, which is people just complaining about bias, not because they wanted the systems to be less biased, but because they wanted it to be biased in their direction.

But I think as I’ve thought about this more, I actually think this is a really important episode in the trajectory of AI, not because it shows that Google is too woke or they have too many DEI employees or whatever.

But it’s just a very good, clear lesson in how hard it is for even the most sophisticated AI companies to predict what their models will do out in the world. This is a case of Google spending billions of dollars and years training AI systems to do a thing and putting it out into the world and discovering that they actually didn’t know the full extent of what it was going to do once it got into users’ hands.

And there’s admission on their part that their systems really aren’t good enough to do what they want them to do, which is to produce results that are helpful and useful and non-offensive.

Right. So I wonder, Kevin, what you think would have been the better outcome here, or what would have been the process that would have delivered results that didn’t cause a controversy because I have a hard time answering that question for myself.

These models are a little weird in the sense that you essentially just throw a wish into the wishing fountain, and it returns something. And it does try to do it to the best of its ability while keeping in mind all the guardrails that have been placed around it.

And to my mind, just based on that system, I just expect that I’m going to get a lot of stupid stuff. I’m not going to expect this prediction-based model to predict correctly every single time.

So to me, one of the lessons of this has been maybe we all just need to expect a lot less of these chatbots. Maybe we need to acknowledge that they’re still in an experimental stage. They’re still bad a lot of the time. And if it serves something up that seems offensive or wrong, maybe just roll our eyes at it and not turn it into a crisis. But how do you think about it?

Yeah, I would agree with that. I think that we still all need to be aware of what these things are and their limitations.

That said, I think there are things that Google could do with Gemini to make it less likely to produce this kind of result.

Like what?

The first is, I think that these models could ask follow-up questions. If you ask for an image of the founding fathers, maybe you’re trying to use it for a book report for your history class, in which case you want it to actually represent the founding fathers as they were.

Or maybe you’re making a poster for “Hamilton,” in which case, you don’t!

Exactly! Or maybe you’re doing some kind of speculative historical fiction project or trying to imagine as part of an art project what a more diverse set of founding fathers would look like.

I think users should be given both of those options. You ask for an image of the founding fathers. Maybe it says, well, what are you doing with this? Why do you want this?

For a chatbot that’s just returning text answers, it could say, do you want me to pick a personality? Do you want me to answer this as a college professor would or a Wikipedia page? Or do you want me to be your sassy best friend? What persona do you want me to use when answering this question?

Right now these AI language models are built as oracles that are supposed to just give you the one right answer to everything that you ask for. And I just think, in a lot of cases, that’s not going to lead to the outcome that people want.

It’s true. But let’s also keep in mind that it is expensive to run these models and that if something like Gemini were to ask follow-up questions of most of the queries that get inputted into this, all of a sudden, the cost just balloons out of control, right?

So I think that’s actually another way of understanding this. Why is Google rewriting a prompt in the background? Well, because it’s serving a global audience.

And if it is going to be showing you a firefighter, it does not want to assume that it’s going to show you only white male firefighters because maybe you are inputting that query from somewhere else in the world where all of the firefighters are not white, right?

So this feels like, in a way, the cheapest possible way to serve the most possible customers. But as we’ve seen, it has backfired on them.

Yeah, I also think that this prompt transformation thing — I think this is a bad idea. I think this is a technical feature that is ripe for conspiracy theorists to seize on and say, they’re secretly changing what you ask it to do to make it more woke.

I just think if I put something into a language model or an image generator, I want the model to actually be responding to my query and not some hidden intermediate step that I can’t see and don’t know about.

At the very least, I think that models like Gemini should tell you that they have transformed your prompt and should show you the transform prompt so that what the image or the text response you are getting actually reflects.

And that is what ChatGPT does, by the way. When you ask it to make you an image, it will transform your prompt in the background. But then once the image is generated, you can click a little info button, and it will tell you the prompt, which is often quite elaborate. I appreciate this feature.

Look, it’s a really interesting product question because speaking on the ChatGPT site, I can tell you, that thing is much better at writing prompts than I am. To me, this totally blew away the concept of prompt engineers, which we’ve talked about on the show.

Once I saw what ChatGPT was doing, I thought, well, I don’t need to become a prompt engineer anymore because this thing is just very good by default. But there are clearly going to be these tripwires where when it comes to, I think, reflecting history in particular, we want to be much, much more careful about how we’re transforming things.

So how do you think this whole Gemini controversy will result? Will heads roll at the company? Will there be people who step down as a result of this? Is it going to meaningfully affect Google’s AI plans? Or do you think this is just going to blow over?

I expect that in the Google case, it will blow over. But I do think that we have seen the establishment of a new front in the culture war. Think about how long in the past half decade or so we spent debating the liberal and conservative bias of social networks.

And oh, the Congressional hearings that were held about, hey, I searched my name, and I’m a Congressman, and it came up below this Democrats name. What are you going to do about it? And we just had this whole fight about whether the algorithmic systems were privileging this viewpoint or that viewpoint.

That fight is now coming to the chatbots, and they are going to be analyzed in minute detail. There are going to be hearings in Congress. And it really does seem like people are determined not to learn the lesson of the content moderation discussion of the past decade, which is that it is truly impossible to please everyone.

Yeah, I do think we will have a number of exceedingly dumb congressional hearings where people hold up giant posters of AI-generated images of Black popes or whatever and just get mad at them.

I do think some of the fixes that we’ve discussed to prevent this kind of thing from happening are short-term workarounds or things that Google could do to get this thing back up and running without this kind of issue.

I think in the longer term, we actually do need to figure out how the rules for these AI models should be set, who should be setting them, whether the companies that make them should have any kind of Democratic input.

We’ve talked a little bit about that with Anthropics’ constitutional AI process, where they actually have experimented with asking people who represent a broad range of views, what rules should we give to our chatbot? I think we’re going to be talking more about that on the show pretty soon.

But I do think that this is the kind of situation and the kind of crisis for Google that a more Democratic system, when it comes to creating the guardrails for these chatbots, could have helped them.

I think that sounds right. But let me throw another possible solution at you, which is over time, these chatbots are just going to know more about us.

ChatGPT recently released a memory feature. It essentially uses part of the context window for its AI to store some facts and figures about you. Maybe it knows where you live. Maybe it knows something about your family. And then, as you ask it questions, it tries to tailor its answers to someone like you.

I strongly suspect that within a couple of years, ChatGPT and Gemini are going to have a pretty good idea of whether you lean a little bit more Liberal, about whether you lean more Conservative, about whether you’re going to freak out if somebody shows you a non-white founding father or not. And we’re going to essentially have all these more custom AIs.

Now, this comes with problems of its own. I think this brings back the filter bubble conversation. And hey, I only talked to a chat bot who thinks exactly like me.

That clearly has problems of its own. But I do think that it might, at least, dial down the pressure on Gemini to correctly predict your politics every time you use the damn app.

Yeah, I think that’s right. I also worry about Google bringing this technology closer and closer to its core search index. It’s using Gemini already to expand on Search results.

And I just think that people are going to freak out when they see examples of the model as it will continue to do, no matter what Google does to try to prevent this. It will give them answers that offend them. I think it’s a very different emotional response when a chat bot gives you one answer than when a search engine gives you 10 links to explore the thing.

If you search images of the American founding fathers on regular old Google Search Engine, you’re going to get a list of things. And some of what’s at those links might offend you. But you as a user are not going to get mad at Google if the thing at those links offends you.

But if Google’s chatbot gives you one answer and presents it as this oracular answer that is the one correct answer, you’re going to get mad at Google because they built the AI model.

So I just think, in a lot of ways, this episode with Gemini has proven the benefits of the traditional search engine experience for Google because they are not taking an editorial position, or, at least, users don’t perceive them as taking an editorial position when they give you a list of links. But when you give them one answer from a chatbot, they do.

That’s right. So maybe that’s a reason why companies like Google should rethink making their footnotes just the tiniest little numbers imaginable that you can barely even click on with your mouse.

Maybe you want to make it much more prominent where you’re getting this information from so that your users don’t hold you accountable when your chatbot says something completely insane.

All right, so that is what’s going on with Gemini. When we come back, Kara Swisher on her new book, “Burn Book,” and hear if she has some burns for us.

[MUSIC PLAYING]

Kevin, let me share a quick story about our next guest. One time, I was asking her for advice, and she gave me great advice about my career. She always does. And then she wrapped up by looking me up and down, and she said, “But just remember, no matter what happens, you’ll be dead soon.”

[LAUGHS]:

And that’s Kara Swisher in a nutshell, Kara Swisher, legendary journalist, chronicler of Silicon Valley. Kevin, on top of all that, she also founded the very podcast feed that you’re now listening to.

Yes. So today, we’re talking with Kara Swisher. Kara, of course, is the legendary tech journalist and media entrepreneur. She has covered the tech industry since, basically, the tech industry existed. She co-founded the publication, Recode, and the Code Conference.

She used to have a podcast called “Sway” at “The New York Times” and be a “New York Times” opinion columnist. And in a bit of internecine podcast drama, there was a little dust up, if you will, when she left “The New York Times” a few years ago. And the podcast feed that her podcast had used was turned into the “Hard Fork” feed, the very feed on which our episodes now rest.

That’s right. She has feelings about that.

She does.

You may hear them on this very interview.

[LAUGHS]: But that’s not why we’re interviewing her. Kara, in addition to being one of the great tech journalists, is also a friend and a mentor to both of us. She was actually your landlord for many years.

That’s right, a very good landlord. I needed to replace a stove one time. She didn’t even blink. She said, just do it right away.

[LAUGHS]: But that’s also not the reason we’re talking to her. We’re talking to her because she has just written a new book called “Burn Book.” It is a memoir full of stories from her many years covering Silicon Valley and bumping elbows with people like Elon Musk and Mark Zuckerberg. I read the book, and it is a barn burner.

Yeah, this is a book where Kara, who is famously productive, slows down and goes back through decades of history, talking to some of the titans of Silicon Valley, and, I think, chronicles her disillusionment, honestly, with a lot of them.

I think she arrived here and was captivated by the promise of the internet. But as the years have gone on, she’s become more and more disappointed with the antics of some of the people running this town.

Yeah, totally. So I want to ask her about the book, but I also just think it’s a good time to talk to her in general, both to see if we can clear up all this drama around the podcast feeds, finally, but also just to get her take as someone who’s been around this industry for longer than almost anyone I know, about the state of, tech what’s happening in the industry, what’s happening in the media, and the tech media, specifically, and where she thinks we’re all heading.

Yeah. And as for me, I’m just trying to get my security deposit back.

[LAUGHS]:

One note about this conversation, it’s very energetic, and I think that energy inspired Kara to drop a lot of F Bombs. So if you are sensitive to that or with younger listeners, you may want to fast forward through this segment.

Yeah, she used up our whole curse word quota for all of 2024 in a single interview. So just —

Oh, rats!

[LAUGHS]: Oh, dang it! [LAUGHS]

[MUSIC PLAYING]

So maybe that’s affecting me. I can’t stand Coldplay. They’re so whiny. And he’s like, he’s a very good friend of mine. I’m like, oh, sorry. Apologies. But he still sucks.

And so that was — was that advice? I didn’t think he’d close it down because I said it sucks. But he knew it already. I didn’t tell him anything he didn’t know. It was stuff like that.

When we come back, the Supreme Court takes on content moderation.

[MUSIC PLAYING]

Casey, you and I have written a few times over the years about the issue of content moderation on social media.

Yeah, it’s one of the biggest issues it seems like anyone wants to talk about when it comes to the social networks.

And this week is a particularly big week in content moderation land because the Supreme Court of the United States heard arguments for two cases that are directly related to this issue of how social networks can and cannot moderate what’s on their services.

On Monday Supreme Court justices heard close to four hours of oral arguments over the constitutionality of two state laws. One came out of Florida. The other is in Texas.

Both of these laws restrict the ability of tech companies to make decisions about what content they allow and don’t allow on their platform. They were both passed after Donald Trump was banned from Facebook, Twitter, and YouTube following the January 6 riots at the Capitol.

Florida’s law limits the ability of platforms like Facebook to moderate content posted by journalistic enterprises and content, quote, “by or about political candidates.” It also requires that content moderation on social networks be carried out in a consistent manner.

Texas’s law has some similarities, but it prohibits internet platforms from moderating content based on viewpoint with a few exceptions.

Yeah, so this is a really big deal. Right now platforms remove a bunch of content that is not illegal. You’re allowed to insult people, maybe even lightly harass them. You can say racist things you can engage in other forms of hate speech. That is not against the law.

But platforms, ever since they were founded, have been removing this stuff because for the most part, people really don’t want to see it. Well, then along come Florida and Texas, and they say, we don’t like this, and we’re actually going to prevent you from doing it. So if these laws were to be upheld, Kevin, you and I would be living on a very different internet.

Yeah, so I think when it comes to content moderation, and its legal challenges, this is the big one. This pair of lawsuits is what will determine how and if platforms have to change the way that they moderate content dramatically.

Yeah. But Kevin, we want to bring in some help to get through the legal issues here today.

Yes, so we’ve invited today an expert on these issues. This is Daphne Koller. Tell us about Daphne.

Daphne is the person that reporters call when anything involving internet regulation pops up. She is somebody who has spent decades on this issue. She’s currently the director of the program on Platform Regulation at Stanford’s Cyber Policy Center.

She has done a lot of great writing on these cases, in particular, including a couple of incredibly helpful FAQ pages that have helped reporters like me try to make sense of all of the issues involved.

Daphne also formally submitted her own views to the Supreme Court in an amicus brief that she helped write and file on behalf of political scientist Francis Fukuyama.

Yeah, so Daphne is opposed to these laws, we should say. She believes that they are unconstitutional and that the Supreme Court should strike them down. But this is not a view she came to lightly or recently.

She’s been working in the field of tech and tech law for many years. We’ll link to her great FAQs in the show notes. But today, for a breakdown of these cases and how she thinks the Supreme Court is likely to rule, we wanted to bring her on. So let’s bring in Daphne Koller.

[MUSIC PLAYING]

Daphne Koller, welcome to the show.

Thank you. Good to be here.

So I want to just start — can you just help us lay out the main arguments on either side of these cases. What are the central claims that Texas and Florida are using to justify the way that they want to regulate social media companies?

So it’s not that far away from the basic political version of this fight. The rationale is these are liberal California companies, or they were liberal California companies, and they’re censoring conservative voices, and that needs to stop.

My understanding is that this is probably the only Supreme Court case in the history of the Supreme Court that had its origins in a “Star Trek” subreddit. Can you explain that whole thing.

So this isn’t literally from that case. So Texas and Florida passed their laws. The platforms ran as fast as they could to courts to get an injunction so the laws couldn’t be enforced.

But a couple of cases got filed in Texas. And the most interesting one — I thought there was just one. I think now there are two, actually. But the most interesting one is somebody who posted on the “Star Trek” subreddit, that Wesley Crusher is a soyboy. I had to look up what soyboy means. It’s junior cook or something.

People often call us soyboys.

That’s like a conservative slur meaning weakling, I think.

Yes.

Yeah.

Yeah, as I sit here drinking my green juice.

But at least it’s not soy milk.

That’s right, right.

So the moderator — it wasn’t even Reddit. The moderators of that subreddit took that down because of some rule that they have.

It’s deeply offensive to members of the “Star Trek” community.

And the soyboy community.

And the soyboy community, yeah.

And the person — I’m going to guess it’s a guy — sued, saying this violates the obligation in Texas law to be viewpoint neutral. And it’s a useful example because it’s such a total real world content moderation dispute about some dumb crap.

But the question of what does it mean to be viewpoint neutral on the question of whether “Star Trek” characters are soyboys helpfully illustrates how impossible it is to figure out what platforms are supposed to do under these laws.

Exactly. You take this very silly case, you extrapolate it across every platform on the internet, and you ask yourself, how are they supposed to act in every single case. And it just seems like we would be consumed with endless litigation.

So you just returned from Washington, where these cases were being argued in front of the Supreme Court. Sketch the scene for us because I’ve never been. What’s it like?

So you start out — if you’re me, you pay somebody to stand in line overnight for you. Because I’m old, I’m not going to do that shit.

But you really — someone had to stand in line overnight for this.

I had somebody there from 9:00 PM, and he was number 27 in line, and they often let in about 40 people.

How do you find these people to just stand in line?

Skiptheline.com.

Wow!

Great tip for listeners.

I learned something today.

Rick did a great job.

Shout out to Rick!

Anyhow, so you stand around in the cold for a long time, then they let you in stages, one of which — the best part definitely is you stand in this resonant, beautiful marble staircase. And a member of the Supreme Court police force explains to you that if you engage in any kind of free speech activity, you will spend the night in jail.

Very firm and polite.

And it’s also interesting to hear that there is effectively content moderation on everyone who is in the room before they even enter. They say, hey, you open your mouth, and you’re out of here.

Yeah. So the people making these arguments represent NetChoice, which is a trade association for the tech companies. It’s their lobbying group. Who else is opposed to these laws?

So I should say that CCIA, which is a different tech trade association, is also a plaintiff, and they always get short shrift because they’re not the first named party. But a whole lot of individual platforms filed or free expression-oriented groups filed, lots of people weighing in who are interested in different facets of the issue.

I see. And for those of our listeners who may not be American or may not have much familiarity with how the Supreme Court works, my understanding is, in these oral arguments, the Justices rain questions down on the attorneys. They try to answer them as best they can.

Then they go away and deliberate and write their opinions. So we don’t actually know how they’re going to rule in this case. But did you hear anything during oral arguments that indicated to you which way this case might be headed?

So there’s a lot of tea leaf reading that goes on based on what happens in oral arguments. And usually, that’s the last set of clues you get until the opinion issues, which seems likely to be in June or something like that. In this case, there’s actually another case being argued in March that’s related and might give us some interesting clues.

But from this argument, it was pretty clear that a number of the Justices thought the platforms clearly have First Amendment-protected editorial rights. And it’s not like that’s the end of the question because sometimes the government can override that with a good enough reason.

But it seemed like there was, I think, a majority for that. But then they all got sidetracked on this question of whether they could even rule on that because the law has some other potential applications. They got into a lawyer-procedural-rules fight that could cause the outcome to be weird in some way.

So let me ask about that. To go back to our soyboy example, to me, if of a private business wants to have a website, and they want to make a rule that says you can’t call anybody a soyboy around here, that does seem like the sort of thing that would be protected under the First Amendment. You write your policies under that First Amendment. Why is that not the end of the story here?

Well, so what Texas or Florida would say is that these laws only apply to the biggest platforms, and they’re so important that they’re basically infrastructure now. And you can’t be hurt at all unless you’re being hurt on YouTube or on X or on Facebook. And so that’s different.

Right, yeah.

So what is the argument from the states about why they should be allowed to impinge on this First Amendment right that these platforms say that they have to moderate content however they want to, their private businesses. What do the states say in response to that?

They say the platforms have no First Amendment rights in the first place. That that’s fake, that what the platforms are doing isn’t speech. It’s censorship. Or what the platforms are doing is conduct. Or mostly they just allow all of the posts to flow, so the fact that they take down some of them shouldn’t matter, a lot of arguments like that, none of which are super supported by the case law. But the court could change the case law.

I want to ask you about another conversation that came up during these oral arguments that you referenced earlier, which was which platforms do these laws apply to? There’s some confusion about this.

And it seemed like the Justices had questions about, OK, maybe if we want to set aside for a second the Facebooks and the Xs and the YouTubes, what about an Uber or a Gmail? Maybe there should be an equal right of access there.

So I look at that, and I say, well, that’s a good reason not to pass laws that affect every single platform the same way. But I’m curious how you heard that argument and maybe if you have any thought about how the Justices will make sense of which law applies to what and what might be constitutional and what might not be.

Yeah, so that part of the argument, I think, caught a lot of people, including me, off guard. We did not expect it to go in that direction. But I’m a little bit glad it did.

I think it was the Justices recognizing, we could make a misstep here and have these consequences that we haven’t even been thinking about. And so we need to look really carefully at what they might be.

And in the case of the Florida law, in particular, the definition of covered platforms is so broad. It explicitly includes web search, which I’m a former legal lead for Google web search, full disclosure. And it seems like it includes infrastructure providers like Cloudflare.

So it’s really, really broad, who gets swept in. And I reluctantly must concede. I think the Justices were right to pause and worry about that.

Yeah yeah.

For sure.

Yeah. A lot of the people I saw commenting on the oral arguments this week suggested that this was going to be a slam dunk for the tech companies, that they had done a good job of demonstrating that these laws in Texas and Florida were unconstitutional, and that it sounded after these arguments like the Justices were likely to side with the tech platforms. Is that your take, too?

I think there — I think enough of them — you need five. I think at least five of them are likely to side with the platform, saying, yes, you have a speech right, and, yes, this law likely infringes it. But because of this whole back and forth they got into about the procedural aspect of how the challenge was brought, it could come out some weird ways.

For example, the court could reject the platforms’ challenge and uphold the laws but do so in an opinion that pretty clearly directs the lower courts to issue a more narrowly-tailored injunction that just makes the law not apply to speech platforms.

There are a lot of different ways they could do it, some of which would formally look like the states winning. Although, it wouldn’t, in substance, be the states winning against the platforms that we’re talking about most of the time, the Facebooks, the Instagrams, the TikToks.

Very interesting.

Yeah. So we’ve talked about these laws on the show before, and I think we can all agree that there are some serious issues with them. They could force platforms operating in these states to open the floodgates of harassment, and toxic speech, and all these kinds of things that we can all just agree are horrible.

But there is also an argument being made that ruling against these cases, striking these laws down, could actually do more damage. Zephyr Teachout, who’s a law professor at Fordham, wrote an article in “The Atlantic” recently about these social media laws called “Texas’s social media law is dangerous. Striking it down could be worse.”

She’s basically making the case that if you strike down these laws, you basically give tech giants unprecedented and unrestrained power. What do you make of that argument?

So I read the brief that Zephyr filed along with Tim Wu and Larry Lessig, and it’s like they’re writing about a different law than the actual law that is in front of the court.

And I think their worry is important. If the court ruled on this in a way that precluded privacy laws and precluded consumer protection laws, that would be a problem.

But there are a million ways for the court to rule on this without stepping on the possibility of future better federal privacy laws, for example. It’s not some binary decision where the platforms winning is going to change the ground rules for all those other laws.

So you don’t worry that if this case comes out in the company’s favor that they’re going to be massively empowered with new powers that they didn’t have before?

Well, if the court wanted to do it that way, if there are five of them who wanted to do it that way, then it could come out that way. But I can’t imagine five of them wanting to empower platforms, in particular, that way, and I can’t imagine the liberal justices wanting to rule in a way that undermines the FTC from being able to do the regulation that it does.

A big topic that comes up in discussions of law and tech policy is Section 230. This is the part of the Communications Decency Act that basically gives broad legal immunity to platforms that host user-generated content.

This is something that Conservative politicians and some Liberal politicians want to repeal or amend to take that immunity away from the platforms. This is not a set of cases about Section 230. But I’m wondering if you see any ways in which the way that the Supreme Court rules on this could affect how Section 230 is applied or interpreted?

Well, you might think it’s not a case about 230 because they agreed to review a First Amendment question, full stop. But the states managed to make it more and more like a case about 230, and multiple justices had questions about it.

So it won’t be too surprising if we get a ruling that says something about 230. I really hope not because that wasn’t briefed. This wasn’t what the courts below ruled on. It hasn’t really been teed up for the court. It’s just they’re interested in it.

There are two ways that 230 runs into this. I think one will be too in the weeds for you. But the more interesting one is lots of the Justices have said things like, look, platforms, either this is your speech and your free expression when you decide what to leave up, or it’s not, and you’re immunized.

Pick one. How can it possibly be both? And the answer is no, it can definitely be both. That was the purpose of Section 230, that Congress wanted platforms to go out there and have editorial control and moderate content. Literally, the goal was to have both at once.

Also, if the platforms have First Amendment rights in the first place, It’s not like Congress can take that away by passing an immunity statute. That would be a really good one weird trick, and I’m glad they can’t do that.

So there are a lot of reasons that argument shouldn’t work, but it’s very appealing, I think, in particular, to people whose concept of media and information systems was shaped in about 1980.

If the rule is you have to be either totally passive, like a phone company, and transmit everything, or you have to be like “NBC Nightly News.” and there are just a couple of privileged speakers, and lawyers vet every single thing they say, then you’re going to get those two kinds of communication systems.

You’ll get phones, and you’ll get broadcast, but you will never get the internet and internet platforms and places where we can speak instantly to the whole world but also have a relatively civil forum because they’re doing some content moderation.

Right. It almost sounds like there’s a downside to having the median age of a Supreme Court justice being 72.

I don’t know what the real age is. I’m sure I’ll do a pick up about that later.

Now, Kevin, do you want to tell her who wrote the 230 question?

[LAUGHS]: What? You’re going to out me like this?

I’m going to out you.

So this was a great question that I, unfortunately, did not write, but the Perplexity search engine did because I gave it the prompt, “Write 10 penetrating grad student-level questions for a law and policy expert about the NetChoice cases.

In fairness, I did think it was a pretty good question.

It was a very good question. So yeah, wow, you’re really you’re really doing me dirty here. I was going to get away with that.

Look, we wrote the rest of the questions. We just wanted a little help to make sure left no stone unturned.

Yeah, and it was a pretty smart question. Smarter than I would have come up with.

And let’s say, the answer is way better than the question.

Yes, that’s true.

A student of mine sent me a screenshot of something he got from ChatGPT. He’d asked for sources on some 230-related thing, and it cited an article that it pretended I had written, which did not exist, the Twitter files and Section 230 that was in a nonexistent journal called “The Columbia Journal of Law and Technology.” It looked very plausible.

I’m comfortable being cited in things I didn’t write as long as they were good and in prestigious journals. You know what I mean?

I loved your submission to “The New England Journal of Medicine.”

(LAUGHING) Thank you so much!

It was really good.

It saved a lot of lives.

So Daphne, we’ve talked about how the Court will or may handle these cases. But I’m also curious how you think they should handle this. You and some other legal experts filed an amicus brief in this case, arguing for —

Actually, let’s settle this once and for all. Is it amicus or is it amicus, Daphne?

It’s both end.

OK, great

Wow!

And some people say the plural, amici.

Oh!

Ooh!

I ordered that in an Italian restaurant once.

I think I saw him DJ in Vegas.

All right.

Can you just articulate your position in that brief about how you think the court could and maybe should handle this?

Yeah, so this is not how the parties have framed it. This is some wonks coming in and saying, you framed it wrong. But I do actually think they framed it wrong.

So there’s a standard set of steps in answering a First Amendment question. You ask, did the state have a valid goal in enacting this law, and does the law actually advance that goal, and does it have an unnecessary damage to speech that could have been avoided through a more narrowly tailored law?

So in this case, the states say we had to pass this law because the platforms have so much centralized control over speech. Let’s assume that’s a good goal. We say that doesn’t mean the next step is the state takes over and takes that centralized control to impose the state’s rules for speech.

There are better next steps that would be more narrowly tailored, that would be a better means-ends fit and, in particular, steps that empower users to make their own choices using interoperability or so-called middleware tools for users to select from a competing environment of content moderation.

What would this look like? This would be like a toggle on your Reddit app that would say, I want soyboy content, or I don’t want soyboy content content?

So it could look like a lot of different things. But I know you guys have talked to Jay from Bluesky. It could look like what Bluesky is trying to do with having third parties able to come build their own ranking rules or their own speech-blocking rules, and then users can select which of those they want to turn on.

It could look like Mastodon with different interoperating nodes where the administrator of any one node sets the rules. But if you’re a user there, you can still communicate with your friends on other nodes who have chosen other rules. It could look like Block Party, back when block party was working on Twitter. You download block lists that are —

This was an app that basically lets you block a bunch of people at once.

Yeah. So it could look like a lot of things, and all of them would be better than what Texas and Florida did.

Right.

I wonder if you can sort of steelman the argument on the other side of this case a little bit. I was going through this exercise myself because on one hand, I do think that these laws are a bad idea.

On the other hand, I think that the tech platforms have, in some cases, made their own bed here by being so opaque and unaccountable when it comes to how they make rules governing platforms and, frankly, spending a lot of time obfuscating about what their rules are, what their process is, doing these fake oversight boards that actually have no democratic accountability.

It’s a kangaroo court. Come on. And I think I’m somewhat sympathetic to the view that these platforms have too much power to decide what goes and what doesn’t go on their platforms.

But I don’t want it to be a binary choice between Mark Zuckerberg making all the rules for online speech along with Elon Musk and other platform leaders and Greg Abbott and Ron DeSantis doing it. So I like your idea of a middle path here.

Are there other middle paths that you see where we could make the process of governing social media content moderation more democratic without literally turning it over to politicians and state governments?

It’s actually really hard to use the law to arrive at any kind of middle path, other than this kind of competition-based approach we were talking about before.

The problem is what I call lawful but awful speech — a lot of people use that — which is this really broad category of speech that’s protected by the First Amendment, so the government can’t prohibit it, and they can’t tell platforms they have to prohibit it.

And that includes lots of pro-terrorist speech, lots of scary threats, lots of hate speech, lots of disinformation, lots of speech that really everybody across the political spectrum does not want to see and doesn’t want their kids to see when they go on the internet.

But if the government can’t tell platforms they have to regulate that speech people morally disapprove of but that it’s legal and First Amendment protected, then their hands are tied.

Then that’s how we wind up in this situation where instead, we rely on private companies to make the rules, that there’s this great moral and social demand for from users and from advertisers. And it’s extremely hard to get away from because of that delta between what the government can do and what private companies can do.

Well, some people have described our podcast as lawful but awful speech, so I hope that we will not end up targeted by these laws. Daphne Koller, thank you so much for joining us. It’s really a pleasure to have you. Thanks for having me.

[MUSIC PLAYING]

“Hard Fork” is produced by Rachel Cohn and Davis Land. We’re edited by Jen Poyant. This episode was fact checked by Caitlin Love.

Today’s show was engineered by Chris Wood. Original music is by Diane Wong, Marion Lozano, Rowan Niemisto, and Dan Powell. Our audience editor is Nell Gallogly. Video production is by Ryan Manning and Dylan Bergersen.

If you haven’t already, check us out on YouTube at youtube.com/hardfork. Special Thanks to Paula Szuchman, Pui-Wing Tam, Kate LoPresti, and Jeffrey Miranda. You can email us at hardfork@nttimes.com with all your sickest burns.

Please invite us to your Willy Wonka themed events, too.

[THEME MUSIC]