F-Secure’s AI reads mean tweets to fight abuse and trolls
Researchers at F-Secure’s Artificial Intelligence Centre of Excellence and the University of Crete’s Forth-IS Institute have developed a novel method for categorising tweets that they hope will, in future, help platforms such as Twitter clamp down on bad behaviour and deal more effectively with abuse, harassment and other forms of malicious activity.
Working under the auspices of F-Secure’s Project Blackfin, researcher Andy Patel and PhD student Alexandros Kornilakis conducted experiments on replies to US president Donald Trump and other US politicians, including the field of Democratic candidates vying to take Trump on in the November 2020 election.
Patel said that the “torrential downpour” of content on social media gave bad actors cover to spread misinformation, hoaxes, lies, scams and fake news, and the inability of sites to stop this kind of behaviour was creating a marketplace for likes, views, subscribes, reviews, and fake accounts.
In their paper, A New, novel method for clustering tweets, Patel and Kornilakis set out a new method of clustering – the process of using machine learning to group phrases or passages into buckets based on their topic.
They developed a methodology that involved processing captured data, converting tweets into sentence vectors, combining said vectors into meta-embeddings, and then creating node-edge graphs using similarities between calculated meta-embeddings, from which the clusters were then derived.
Patel and Kornilakis’ tested their methodology around multiple events, including the 2019 UK General Election which set a new high-water mark for abusive behaviour online. The bulk of their research, however, centred on over a million replies to tweets sent by Trump, the Democratic Party candidates, and congresswoman Alexandra Ocasio-Cortez.
According to Patel, the online mentions of such politicians present an extreme version of what the average Twitter user may have to deal with. Inevitably, they receive lots of engagement, and it usually skews extremely positive or negative.
Patel and Kornilakis classified the mentions by identifiers such as subject-verb-object and overall sentiment and then used their methodology to build an average sentiment score. Based on these identifiers, posts were classified as positive, negative or toxic.
In the case of the Democrats, the most common negative tweets included terms such as “you are an idiot/moron/liar/traitor”, “you will never be president” and “Trump will win”. More positive themes included “we love you”, “you got this” and “you have my vote”.
The little-fancied Andy Yang received by far the most positive replies, followed by Bernie Sanders and Amy Klobuchar, while Alexandra Ocasio-Cortez and Elizabeth Warren received the most toxic posts.
However, when compared to Trump, none of the Democrats attracted as much toxicity, with the most common themes including “you are an idiot/liar/disgrace/criminal”, “you are not our president”, “you have no idea/you know nothing”, “you should shut up” and “you can’t stop lying”. Many also included references to Vladimir Putin. Trump’s positive mentions tended to include themes such as “God bless you” and “we love you”.
Patel hopes the methodology can be used to help reduce the misuse of Twitter by drawing attention to problematic content before it generates traction. This could prove particularly useful in countering life-threatening misinformation, he said.
“For instance, our methodology automatically identified and grouped tweets pushing a hoax that the Australian bush fires had been caused by arsonists,” said Patel.
“Additional research is necessary, but with some more development there could be a range of potential applications. This methodology could be used for automated filtering or removal of spam, disinformation and other toxic content. This could be done by assigning quality scores to accounts based on how often they post toxic content or harass users.”
The researchers have set up a website for interested individuals to explore the data, and the project’s code can be found on Github.
Project Blackfin was set up in 2019 at F-Secure’s Helsinki headquarters, and is described as a research programme dedicated to the development of decentralised AI for cyber security. The project researchers also hope to take artificial intelligence (AI) to the next level by challenging the common misconception that AIs should mimic human intelligence.
source computerweekly
Industry: Cyber Security
Latest Jobs
-
- Account Director | Cyber Security Consulting | UK - South East
- London
- N/A
-
Account Director | Cyber Security Consulting - Financial Services | UK - South East. New Role due to Growth We are looking for an experienced Account Director to develop and expand existing relationships across the financial services sector, working with investment firms, asset managers, private equity groups and strategic partners to deliver intelligent cyber consulting and a bespoke Cyber product offerings. You will act as a trusted advisor, helping organisations strengthen digital resilience, manage third-party and regulatory risk and adopt a proactive approach to cyber assurance. Key Responsibilities Manage a defined portfolio of financial clients, understanding business priorities and aligning tailored cyber solutions. Drive new client engagement while nurturing existing partnerships through a consultative, long-term approach. Present the benefits of advanced cyber services including threat intelligence, vulnerability management, incident readiness, and continuous risk monitoring. Collaborate with technical and delivery teams to ensure smooth engagement from proposal through to implementation and ongoing support. Prepare proposals, negotiate commercial terms, and clearly articulate value and business outcomes. Build trusted relationships at senior and board level. Ideal Profile Strong background in cybersecurity, consulting, or risk management within financial services. Skilled communicator with proven success managing and growing key accounts. Able to translate complex technical insight into commercial and strategic value for clients. Confident engaging with senior stakeholders and decision makers. Please note: Sponsorship is not available.
-
- SOC Analyst- Level 2- Hybrid Greater London
- London
- N/A
-
SOC Analyst- Level 2- Hybrid Greater London New opportunity created through continued growth. We’re looking for a SOC Analyst (Level 2) to strengthen a growing managed security team. You’ll work hands-on with Microsoft Sentinel and Defender XDR, investigating alerts, responding to incidents, and helping improve how clients stay protected. This role is ideal for someone who enjoys unravelling security events, thinking critically under pressure, and making a real difference day to day. What you’ll do · Investigate and respond to security activity across SIEM and endpoint tools · Analyse network and log data to uncover real threats · Support automation initiatives to streamline response processes · Help maintain visibility, data flow, and performance across SOC platforms What you’ll need · Practical experience using Microsoft Sentinel and Defender XDR · Confident working with KQL or similar query languages · Understanding of attacker tactics and response techniques · SC-200 certifications would be nice. · Experience supporting multiple customer environments Please note: Sponsorship is not available.
-
- Senior SOC Engineer - UK - New role due to growth
- London
- N/A
-
Senior SOC Engineer – New role due to growth We are hiring a Senior SOC Engineer to take the lead across security operations for a growing managed service. You will lead detection, response and onboarding activity across multiple clients, helping shape how the SOC evolves. Expect variety; from fine-tuning alerts and threat hunting to supporting customers and mentoring junior analysts. What you’ll bring · Strong experience across SIEM, EDR, and threat detection tools · Confident working with customers in a managed service environment · Skilled in scripting or query languages such as KQL or PowerShell · Knowledge of frameworks like NIST, ISO27001, MITRE ATT&CK · Calm communicator with a problem-solving mindset · Experience with Azure Lighthouse or delegated access models · Prior involvement in automation or SOC improvement projects Location: South East England- Hybrid role Please note: Sponsorship cannot be offered now or in the future.
-
- SENIOR Cyber Risk Consultant. Cyber Risk consultancy - the right way. UK. Hybrid - Remote first
- United Kingdom
- N/A
-
Senior Cyber Risk Consultant needed. New position due to growth Seeking a passionate Cyber Security Risk Consultant who enjoys helping clients make a different to their business. Dedicated training budgets, Unlimited holiday structured career path, Work life balance guaranteed Cyber Risk consultancy done the right way. A successful individual will have experience working with clients to identify business cyber security risk. This is a remote first opportunity which means you will spend the majority of your time working remotely. You will however spend some time meeting clients as well as meeting up with the team on a monthly basis.. Some of the nice to have certifications. CRISC, ISO27001 Lead implementer, CISA, CISM, CISSP Unable to offer Visa sponsorship now or in the future.