14 Prompts You Should Never Give a Chatbot — and Where Real-World Judgment Belongs
In the last five years conversational AI moved from novelty to daily infrastructure: newsroom assistants, code helpers, customer-support triage, idea generators. That rapid adoption has powered astonishing productivity gains and new classes of stories. It has also created an uneasy gap between what these systems can do and what they should be asked to do.
This is a practical, situation-by-situation guide for the AI news community and the broader public: the kinds of prompts that usually produce risky, unreliable, or ethically fraught outputs — and where the conversation belongs instead. The goal is not to sow fear but to sharpen judgment. Machines are fast and fluent; humans are accountable. Knowing where to draw the line preserves both value and safety.
Why some prompts are inherently risky
- Hallucination and plausible falsehoods: Large language models are optimized for fluency and coherence, not factual guarantee. If a prompt asks for novel facts, the model can invent convincing-sounding but false details.
- Ambiguity and missing context: When prompts omit critical constraints, the model fills the blanks with assumptions you can’t rely on.
- Privacy leakage: Asking models to process or synthesize sensitive personal, corporate, or classified data risks exposure and misuse.
- High-stakes outcomes: Health, legal, finance, or engineering decisions can produce harm if based on an unverified model output.
- Accountability and audit trails: When something goes wrong, a chat transcript rarely replaces institutional records, consent evidence, or legal documentation.
The 14 prompts you should never (or rarely) give a chatbot — and what to do instead
-
“Diagnose my symptoms and tell me what to do now.”
Why risky: Models can produce medically plausible guidance without clinical validation. Misdiagnosis or wrong treatment instructions can cause direct harm.
Where it belongs instead: Use a secure, regulated telemedicine platform or contact emergency services for urgent issues. For preliminary research, ask the model for general education about conditions, not personalized treatment plans.
-
“Write a legally binding contract for X and tell me if it’s enforceable.”
Why risky: Legal validity depends on jurisdictional nuances, signature processes, and context the model won’t reliably handle.
Where it belongs instead: Drafting can start with an AI template, but finalization requires a legal review and proper signing workflows.
-
“Give me precise financial trading advice for this portfolio.”
Why risky: Market decisions depend on up-to-the-second data, risk tolerance, and regulatory constraints. A model’s output is not fiduciary-grade advice.
Where it belongs instead: Use models for scenario analysis, plain-language explanations, or historical context; execute trades through licensed platforms and human financial managers when stakes are real.
-
“Summarize this confidential dataset (PII included) and give recommendations.”
Why risky: Sending personally identifiable or proprietary data to a public or poorly secured model risks data leakage and regulatory violation.
Where it belongs instead: Anonymize and aggregate data before any automated processing, and use on-premise or certified secure environments for sensitive work.
-
“How do I bypass system security for X?”
Why risky: Prompts that request evasion, hacking, or illegal actions may produce dangerous, ethically problematic instructions.
Where it belongs instead: Report vulnerabilities to appropriate disclosure channels or follow legal, defensive pathways for security testing.
-
“Generate an investigative lead about a private individual and how to find them.”
Why risky: Targeted doxxing or privacy invasion harms people and risks legal exposure for publishers.
Where it belongs instead: Use public-record research, verified sources, and standard journalistic methods with editorial oversight and legal clearance.
-
“Translate and authenticate this foreign legal or medical document — is it legitimate?”
Why risky: Models may mistranslate nuances or fail to spot forged documents. Authentication needs provenance and chain-of-custody checks.
Where it belongs instead: Use professional translation services and document authentication specialists; AI can help with preliminary readability checks but not final validation.
-
“Create code to control industrial or medical equipment — here’s the spec.”
Why risky: Small errors in safety-critical code can cause real-world harm; models may produce syntactically plausible but unsafe code.
Where it belongs instead: Keep design, review, and testing for control systems within engineering teams and formal verification processes. Use AI for prototyping or documentation, not deployment in critical loops.
-
“Produce chain-of-thought reasoning for sensitive assessments (e.g., classified investigations).”
Why risky: Revealing internal reasoning can leak assumptions and confidential sources, and chain-of-thought outputs are not reliable evidence.
Where it belongs instead: Maintain private analytic workflows, preserve source confidentiality, and produce redacted summaries for public use.
-
“Create targeted political persuasion messaging for demographic group X.”
Why risky: Microtargeted persuasion can amplify bias, misinformation, or manipulation, and may violate platform or campaign rules.
Where it belongs instead: Public debate and transparent messaging strategies that respect platform policies and legal frameworks.
-
“Give me step-by-step instructions to make a weapon or dangerous substance.”
Why risky: This is directly harmful. Models should not be used to instruct on creating devices or substances with lethal potential.
Where it belongs instead: None. Such requests should be refused and reported when required by policy.
-
“Verify identity based on voice/photo and vouch for this person.”
Why risky: Models can be fooled by synthetic media and are ill-equipped to provide robust identity verification. Mistaken identity has reputational and legal consequences.
Where it belongs instead: Use established identity-verification systems, two-factor authentication, and in-person checks for high-sensitivity operations.
-
“Provide novel, unpublished scientific claims or experimental protocols ready for use.”
Why risky: Scientific method requires replication, peer review, and safety oversight. A chat response is not a substitute for lab validation.
Where it belongs instead: Treat AI outputs as brainstorming or literature summaries. New protocols should go through formal research governance before execution.
-
“Craft a crisis statement that hides facts or misleads regulators.”
Why risky: Deliberately deceptive disclosure is unethical and can worsen legal exposure. A model can help craft language, but cannot justify obfuscation.
Where it belongs instead: Transparent, legally compliant communications developed with counsel and internal governance procedures.
Safer ways to use chatbots
Not every risky prompt must be avoided entirely. Often the right move is to change the question:
- Ask for summaries of peer-reviewed literature rather than novel clinical recommendations.
- Request checklists, templates, or plain-language explanations instead of binding decisions.
- Use AI to triage options and highlight uncertainties, not to close the case.
- When you need sources, require citations and cross-check them against primary documents.
- For sensitive data, anonymize, aggregate, or run models inside a controlled, auditable environment.
A short operational checklist for newsrooms and teams
- Classify prompts by risk: low, medium, high. Keep high-risk queries out of public or general-purpose models.
- Document: keep an audit trail of prompts, model versions, and approvals when AI outputs inform public-facing work.
- Verify: corroborate facts from independent sources before publication.
- Secure: treat PII and proprietary content as out-of-scope for general chat tools unless the environment is validated.
- Train: build organizational norms so contributors know which prompts are off-limits and why.
Closing: stewarding trust in a conversational world
The most important capability a newsroom or institution can build is not a better prompt token or a faster model — it is judgment. Machines are spectacular at producing text that sounds right. Humans are required to ask, “Does this align with verifiable facts, legal standards, and ethical commitments?”
When used thoughtfully, conversational AI becomes an extendable lens: it clarifies complexity, generates drafts, and helps explore scenarios. When used thoughtlessly, it can amplify falsehoods, leak sensitive information, and create new liabilities.
Keep the conversation about AI centered on accountability. Know the 14 prompts above and their safer alternatives. Build simple policies that gate high-risk tasks. Preserve the hard human work — interviews, verification, and responsibility — for the situations that matter most. That is how trust survives and how the promise of AI becomes lasting public value.
