Imported from pinboard.in/u:timburks
Links tagged 'datapoisoning'
[2312.04748] Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attacks 🚀
https://arxiv.org/abs/2312.04748
While poisoning attacks have received significant attention in the image domain (e.g., object detection), and classification tasks, their implications for generative models, particularly in the realm of natural language generation (NLG) tasks, remain poorly understood. To bridge this gap, we perform a comprehensive exploration of various poisoning techniques to assess their effectiveness across a range of generative tasks.
Note that this is specifically for natural language generation tasks.
In this section, we demonstrate the effectiveness of our designed data poisoning attacks on poisoning LLMs during fine tuning for two NLG tasks: text summarization and text completion... For both tasks, our attacks reduce the model performance only marginally... Overall, our findings suggest: 1. Increasing the percentage of poisoned training data in general significantly improves the success of the attack, while slightly decreases the stealthiness. 2. For text summarization, full fine-tuning is more susceptible to poisoning attacks than prefix-tuning; and vise versa for text completion. 3. Trigger insertion plays a crucial role in the success and stealthiness of the attack. 4. Hardness of attacks depends on the task.
But first movers don't always have to have great results. "Data Poisoning" is a concept that will be with us for a while.
To the best of our knowledge, this is the first work to investigate and characterize in detail poisoning attacks on NLG tasks.
Poisoning Data to Protect It – Communications of the ACM 🚀
https://cacm.acm.org/news/poisoning-data-to-protect-it/
"It's like they're saying, 'if you ask nicely, we won't break into your house.' How about you just don't break in at all?"
New tools are adding structured noise to images that confounds AI model training.
I suppose that someone somewhere is thinking about ways to defeat this, but what a thing to do with your life.
The article ends with a nice set of links to reviewed articles for further reading.
⟪ recent ⟫ advice aftershokz agents agi ai alienhominid alltheplaces android apimanagement apis apisyouwonthate appengine apple art auden automation avro badshah beastieboys benfolds bios blogging boba bobdylan books breaches breakfast brevity brunopedro bsky buf bullshit cacm cameronblevins capitalism changesets chatgpt cherylwaters christophkern cli cloud cloudrun cncf coffee commenting community companies concerts conferences connect cplusplus css dart dartmouth dashboards data databases datalakes datapoisoning debugging defunkt design devex devsite diet dirtywave documentation easteregg eda editions editors edm eks empire endpoints engineering envoy events faith family finch flaxseed frost fruit ftc gallbladder games gateway gateways gcp geekbench geo girard github gloo go google googlemaps gorilla gregorymone grpc grpcweb hacking health healthchecks heartworms help heresy hichord history homelab http hype hypebusting iceberg ideas imgoing india innerengineering inonshkedy integrations interviews iusethis jamesmurphy java jennifergovola jokes json juliaangwin k8s kafka kagi kaitenzushi keithharing kelseyhightower kentstate kexp kiosks kubernetes law lcdsoundsystem licenses linkblogs llms localfirst locations lucagalente lyrics m8 malloryhaigh martinkleppman matduggan materialdesign mccarthy meetups meridethwhittaker meta microsoft middleware minipcs minneapolis minsky museum music nat networking nginx npr nutrition nyt openapi opensource openstreetmap operators oreilly otobokebeaver overture pancakes performances pescatarian peterdenning pharisees pinboard pinkpantheress platformcon platformengineering platforms podcasts poetry portland portugaltheman postgis postideas privacy production productreviews programming prost protobuf protocolbuffers protos pubsub python quality ransomware raphaelpinson recipes repos rss rtree rubrik rust saas sabotage sadhguru santaclarauniversity score scrapers scu sdks seahorse search security sfmoma signal snl snowflake software songs soup spotify spotifyengineering sqlite startups steelydan storage strawberries styleguides sudorandom super73 sushi synthesizer synthhistory teams teensy tiles timbowmanjr timburks tinydesk toddlyons tonic trackers travel turing unkey usps vanta vegan via:license victortangermann videos vulnerability walking web webarchive webinars weezer wikimedia williamdalrymple wix workflows workouts yoga youtube zed zombiezen