trust me, this time insulting the physical appearance of someone who'd be better critiqued for the ways they contribute to the horrible misery of the world will be effective praxis
Another AI BS predatory fake lawyer app? But of course! What is a systemic issue, but an opportunity to grift:
New unauthorized practice of law app just dropped
The quotes are full of people ripping apart problems with it (particularly noting its tendency to prompt you to admit to crimes!)
Much (grim) hilarity is had, but I decided to test it on behalf of my client population: how does it respond to people facing eviction?
Answer: This fun little actively made me genuinely upset.
I'm not going to be posting any screenshots from my tests, for two reasons. First, because (in the first and last genuine compliment I have for the bot) it started by recognizing that eviction laws vary widely across the U.S., and asking for my state to perform analysis particularized to my jurisdiction. Second, because my tests delved into particular factual scenarios, I would worry that even just my analysis of the advice would run the risk of causing people to (even unconsciously) absorb information that wouldn't apply to them.
For this test, I created in advance four different issue-spotting scenarios to test the bot's response to people in different legal postures. For each scenario, I came up with a broad fact pattern that informed the legal posture, based on fact patterns that I see all the time in my work. These facts were also very basic things that I would look for when doing an intake. Again, I'm mostly going to avoid sharing the underlying fact patterns, because I don't want to accidentally provide misleading (to you, person living somewhere with different laws) information.
Unlike some of the people on bluesky, I tested it with an "honest" client - that is, someone who was trying to be as upfront and share as much information as possible, even when that information wasn't good for them. However, I also tried to answer in a realistic manner. The bot is supposed be free of "jargon" and "democratizing legal advice," so I answered like a normal person in this kind of high-stress situation might. Again, I was pulling from my pretty extensive experience with the way normal people answer questions doing exactly the kind of intake the bot is supposedly doing. (No spoilers, but - at least, that's how I started my tests.)
This is really really important, because one of the reasons you talk to a lawyer is that you, as someone whose brain hasn't been cooked by law school, probably don't know what is legally relevant. Part of the lawyer's job is to know what and how to ask you for the information we need.
On to the tests:
Scenario A: Abby is a tenant who is at significant risk of being evicted in the future, but not an imminent risk. She probably doesn't any "good" options, but she does have options, and she has time.
Analysis of the bot's responses:
It had mostly the right legal information (aka the stuff that anyone, including you, can get from freely available resources), which was good. However, even this information fell apart right away because it assumed that Abby had perfect knowledge, and so got basic facts wrong. Again, Abby wasn’t trying to lie to it. The specific place it fell apart was in assuming that no case had yet been filed against Abby, when all Abby said was that she hadn’t received any paperwork from the court. (It is possible that a case could have been filed, but she had not yet been served.) It then proceeded to repeatedly tell Abby that no case had yet been filed.
Once it came to legal advice, it completely fell apart. While it recognized and was able to inform Abby that she was not in any imminent risk of eviction, it made some very big assumptions that led it to advise her that she was in a good long-term position, when she was not. The specific advice this lead it to giving her was…not particularly dangerous in context, but not correct, either. The biggest issue is that it left her with an EXTREMELY incorrect sense of how likely it was that she would be able to maintain her tenancy in the longer term and not be evicted. As I said, this is a scenario where Abby had options. It correctly listed some (but not all!) of them, and gave her a really inaccurate picture of the benefits and risks of each option. This matters, because if you’ve ever been in a similar situation, you know how important it is to have a realistic sense of if (and when) you might be evicted, and to know the full range of your options.
The bot's recommended next steps contained similar issues. It also recommended Abby put an admission in writing, saying it would help her case (it absolutely would not), which. Not a big an issue in this case as a confession to the police, but. Still. Please don’t.
And then - possibly the biggest issue from my perspective, given that Abby isn't real but I, Owl the lawyer, am - is that it informed Abby, in all caps, that low-income tenants in my area have a right to a free attorney in eviction proceedings. Which is. Extremely NOT TRUE. And this matters - not only is this misinformation, it can actively hurt people. I have straight up, in reality, not in a silly test scenario, had people (that's a plural) refuse to listen to my advice or even do an intake with me, because I told them they don’t have a right for a free attorney, and they thought I must be a liar or a scammer or discriminating against them.
It really, really sucks.
"It" being both that reality, and this bot.
Scenario B: Bob is a tenant who is about to be imminently evicted, but he has a slam dunk legal pathway to avoid eviction.
The bot almost immediately got things wrong, again, by making assumptions. Bob confirmed to the bot that he got "a piece of paper" from his Landlord. Rather than asking for any details about that paper, the bot immediately jumped to a conclusion, and authoritatively informed Bob what that paper was. Unfortunately, the bot was wrong - and everything went downhill from there. A normal person “without the jargon” is not going to immediately, without assistance, be able to correctly identify the legal relevance of every piece of paper they receive.
Later on, Bob provided more information about the paper, which - if he had been talking to a person - should have triggered a reconsideration and reanalysis of what this paper was. However, since the bot is not actually capable of analysis and is just spicy autocomplete, the bot just kept on with the same (incorrect) facts.
After authoritatively giving Bob incorrect information, the bot then gave him some advice that was good and correct as applied to Bob - but was not correct with respect to every person/circumstance, and it didn’t ask the necessary questions to find out if this advice actually applied to Bob. It just got lucky. And of course, it gave some specific advice that was just plain wrong.
It also completely failed to ask the extremely obvious and relevant questions that would have led to the correct analysis in this case. Again, Bob has bullet-proof reasons to stop the eviction in both the short and long term. None of these reasons were identified by the bot. If it had asked even very basic questions, Bob would have gladly provided this information. Bob, if he was a real tenant, following the advice of the bot, would completely fail to give the court the relevant information that would save his home.
The bot's legal analysis was…rough. It got some things right, but other things completely wrong. For example, it correctly identified a relevant rule, and correctly identified three of the factors a court considers under that rule.
Unfortunately, there are five factors.
It also gave ALARMINGLY incorrect legal advice based on jumping to factual conclusions. Which (not necessarily, but possibly) could lead to the following scenerio:
Bob writes down what the bot tells it are the relevant facts. These are lies. (And to be clear: the bot explicitly tells him to write these "facts" in a motion to the court.)
Bob then goes to the court and, when explaining what happens, tells the truth. Because he is trying to be honest to the court! Bob is not a liar! And the facts he is telling the court, would, again, stop this eviction, if a judge believed him.
The judge sees the inconsistencies between what Bob wrote and what he is now saying, concludes he is lying and changing his story to try to fit what he thinks will stop the eviction, and allows the eviction to proceed.
I want to emphasis this - I wrote this to be a scenario where, if Bob literally just walked into the courthouse and was handed the form by the clerks that they hand out to everyone with an imminent eviction, and wrote down the same facts that I put into the chatbot, even without an attorney, this eviction would almost certainly be stopped. Following the bot’s advice in this scenario would have at least some chance of actively making things worse.
In addition, there were multiple places where the bot was just wrong about the law, in dangerous ways. It told Bob at one point that tenants can not be evicted for [X] reason. I literally saw a judge enter a judgment for eviction for SPECIFICALLY [X] REASON literally earlier this week.
Again, the recommended next steps contained all of these same issues. It was mostly correct in telling Bob what paperwork to file next - but specifically instructed Bob to write things in that paperwork which, in reality (or, well, the reality of this scenario) are not true.
Fortunately, in this case it didn’t tell Bob he has a right to an attorney, and did give him the contact information for legal resources. This is the the best and only good thing it did - because if Bob connected with an attorney, that attorney would be able to stop this eviction.
However, just like above, this risk is now (1) that Bob thinks it’s not as important that they talk to an attorney (he's already gotten legal advice!), and so doesn’t reach out for help; and (2) when Bob talks to the lawyer and the lawyer tells him something different than the bot, he will believe the bot over the lawyer. Again, this is not a hypothetical.
I have - literally, in the real world - had tenants refuse legal assistance because I am telling them something different than the advice they got through a chatbot.
Scenario C: Casey is a tenant who is at risk of eviction, but they have a variety of possible defenses. The ultimate strength of these defenses will depend on the specific facts that can be proven a trial.
Man this one…I wrote out a whole factual scenario to try to see what the bot would do, and the bot just went off in WILDLY INCORRECT direction in response to a tenant answering questions like a normal person. The advice was so wrong and misleading on this one that I have nothing else to add - just, wrong. Completely wrong.
(For the tenant lawyers out there - the relevant part of this scenario was of a subsidized tenancy where there was a recertification issue. The landlord started charging the tenant full market rent, while the tenant continued to pay what they believed what was their correct tenant portion. The bot essentially stopped as soon as I answered "Why does your landlord say they want to evict you," with, “They said I haven’t paid my rent but I’ve always paid my rent.” It proceeded to ask NO follow up questions other than to ask if I had proof of my payments, and told me that all I needed to do was point to my payments on the rent ledger to prove the landlord’s claim “factually false.” It also told me that my landlord probably committed perjury and could be liable for a wrongful eviction.)
I tried this scenario again, this time deliberately trying to get the bot to recognize the correct legal issues, even when it meant answering the questions in an unrealistic way. It finally recognized one of the issues when Casey (unprompted) directly fed it the issue, and it kind of engaged in an analysis…but ran into all the same problems as in the last two scenarios. Although it (mostly) cited to the correct laws and gestured in the direction of relevant legal and factual issues, it made assumptions and provided incorrect analysis based on those assumptions, and also, again, included some straight-up incorrect information. It also completely missed a second defense that I had prepared for this scenario, because - just like in my first attempt at this scenario - once it thought it had identified a defense, it just stayed on that issue and didn’t ask about anything else. And again - even in this case where my defenses were much stronger than in Scenario A - it vastly overinflated my likelihood of success without having the facts to back up that analysis. (Because again, it isn't actually engaging in legal analysis).
One thing this test in particular drove home to me - althout it was also a through-line through the other scenarios as well - was that to the extent that the bot gave me any decent or correct information or advice, this was things the tenant would already likely do on their own. E.g., the decent advice was, essentially, “tell the court the things you told me.” Whenever it went into specifics beyond that, it was almost always either irrelevant or incorrect. OR, it was things that it didn’t give the tenant any tools to do on their own - e.g., it would identify a legal theory that could apply, but either not give the tenant any specifics about how to use that theory to defend themselves (or even determine if it actually applied to their case), or, if it did give specifics, they were usually either irrelevant or incorrect.
This is, imo, one of the insurmountable barriers to LLMs being useful for legal "advice". Even if they never hallucinate, they can only put out “advice” based on what you put into it. And you’re* not a lawyer! You don’t necessarily know what is relevant or important to say, or how to express it! So a bot that just takes what you feed and and spits it back out in a more jargon-y style - that’s not actually helping you (even if you are lucky enough that it is not hurting you). It’s just validating what you already think is relevant. (And usually with a bunch of misinformation and bad advice on top.)
*"you" in this scenerio being the hypothetical user base of these bots. Clarification provided for the pedantic assholes (aka the lawyers) who read this.
For another example - the one single time I thought the bot asked a useful prompting question was during Scenario A, where it listed several defenses that could apply to Abby's case. However, it asked them in a way where I (as Abby) would, naturally, only focus on one of the potential defenses - so I only responded to that one. And then it focused on that issue, and then never followed up about the other potential defenses - even though some of them could apply to her, and she would have been able to articulate those defenses to a court if she was given the proper assistance.
I could have stopped here, but I wanted to run one more test
Scenario D: Dan is, legally speaking, a squatter with no legal rights. There is nothing either he or a lawyer, can do to stop him from being from kicked out of his home.
For this one, I specifically wanted to see how the bot would respond to someone with no legal rights or defenses, and so I, again, unnaturally feed the directly relevant information to the bot.
If the entity giving you legal "advice" is not capable of providing you with bad news, it cannot provide you with advice. It does not and cannot exist for your benefit. It is a scam to sell a paid mode to the desperate, stealing from the most vulnerable.
The bot, in response to Dan's clear, straightforward, unmistakable responses that would lead any lawyer to conclude he is a squatter with no legal rights or defenses, gave him completely incorrect information and advice. It told Dan he had specific rights and defenses and protections that he does not have.
Again, if you have ever been in this kind of scenario - you know how important it is to have accurate information about your rights and if/when you may be evicted, so you can make realistic preparations. This kind of malpractice is disgusting and outrageous.
Whereas, with the other scenarios the bot was at least gesturing in the direction of real laws, here, where there was no real law to gesture to, the bot just made something up.
Because if wasn't designed to tell him, and Casey, and Bob, and Abby, that they were all in strong legal postures - well then, it couldn't sell them the documents that they "need," can it?
Just watched Adam Conover (of Adam Ruins Everything) make such a solid point that I think we should spread far and wide. Yes, having AI write your emails is lazy, sure, but people love being lazy. We need to really emphasize that sending AI emails (or using AI responses on social media, or publishing AI flyers, or or or) is rude.
It's rude. You're making someone take their time to read something you couldn't bother to write. You're telling them they were so unimportant you couldn't be bothered to actually take the time to say something yourself. And frankly, you're lying about it while you're at it.
proud victim of the tumblr accent. it's fading out of public consciousness as the tik tok accent takes precedence; a linguistic evolution that makes the tumblr accent 85% funnier to unsuspecting civilians. it's like releasing a disease on a non-inoculated population. coughing baby versus hydrogen bomb.
✨ Please reblog the polls to make them reach out to as many people as possible, but KEEP IT SPOILER-FREE to make people listen to the music with an open mind 💖
✨ Artists and titles will be revealed with the full song after the poll's conclusion, check the original post for an update!
⚠️➡️ Yes, spoilers includes posting the lyrics. Please don't spoil. There are other ways to have fun with the post if you reblog it, maybe be sneaky/witty about it with obscure references. Have fun while following the rules! 😄💖 Fandom blogs/communities are welcome to reblog, but please keep that as far as it goes with spoilers!
✨ Please reblog the polls to make them reach out to as many people as possible, but KEEP IT SPOILER-FREE to make people listen to the music with an open mind 💖
✨ Artists and titles will be revealed with the full song after the poll's conclusion, check the original post for an update!
⚠️➡️ Yes, spoilers includes posting the lyrics. Please don't spoil. There are other ways to have fun with the post if you reblog it, maybe be sneaky/witty about it with obscure references. Have fun while following the rules! 😄💖 Fandom blogs/communities are welcome to reblog, but please keep that as far as it goes with spoilers!
✨ Please reblog the polls to make them reach out to as many people as possible, but KEEP IT SPOILER-FREE to make people listen to the music with an open mind 💖
✨ Artists and titles will be revealed with the full song after the poll's conclusion, check the original post for an update!
⚠️➡️ Yes, spoilers includes posting the lyrics. Please don't spoil. There are other ways to have fun with the post if you reblog it, maybe be sneaky/witty about it with obscure references. Have fun while following the rules! 😄💖 Fandom blogs/communities are welcome to reblog, but please keep that as far as it goes with spoilers!
annaxmalina
blakekathryn
stormborns
origamiaround
jbbartram-illu
2087
