F-Secure’s AI reads mean tweets to fight abuse and trolls

Researchers at F-Secure’s Artificial Intelligence Centre of Excellence and the University of Crete’s Forth-IS Institute have developed a novel method for categorising tweets that they hope will, in future, help platforms such as Twitter clamp down on bad behaviour and deal more effectively with abuse, harassment and other forms of malicious activity.
Working under the auspices of F-Secure’s Project Blackfin, researcher Andy Patel and PhD student Alexandros Kornilakis conducted experiments on replies to US president Donald Trump and other US politicians, including the field of Democratic candidates vying to take Trump on in the November 2020 election.
Patel said that the “torrential downpour” of content on social media gave bad actors cover to spread misinformation, hoaxes, lies, scams and fake news, and the inability of sites to stop this kind of behaviour was creating a marketplace for likes, views, subscribes, reviews, and fake accounts.
In their paper, A New, novel method for clustering tweets, Patel and Kornilakis set out a new method of clustering – the process of using machine learning to group phrases or passages into buckets based on their topic.
They developed a methodology that involved processing captured data, converting tweets into sentence vectors, combining said vectors into meta-embeddings, and then creating node-edge graphs using similarities between calculated meta-embeddings, from which the clusters were then derived.
Patel and Kornilakis’ tested their methodology around multiple events, including the 2019 UK General Election which set a new high-water mark for abusive behaviour online. The bulk of their research, however, centred on over a million replies to tweets sent by Trump, the Democratic Party candidates, and congresswoman Alexandra Ocasio-Cortez.
According to Patel, the online mentions of such politicians present an extreme version of what the average Twitter user may have to deal with. Inevitably, they receive lots of engagement, and it usually skews extremely positive or negative.
Patel and Kornilakis classified the mentions by identifiers such as subject-verb-object and overall sentiment and then used their methodology to build an average sentiment score. Based on these identifiers, posts were classified as positive, negative or toxic.
In the case of the Democrats, the most common negative tweets included terms such as “you are an idiot/moron/liar/traitor”, “you will never be president” and “Trump will win”. More positive themes included “we love you”, “you got this” and “you have my vote”.
The little-fancied Andy Yang received by far the most positive replies, followed by Bernie Sanders and Amy Klobuchar, while Alexandra Ocasio-Cortez and Elizabeth Warren received the most toxic posts.
However, when compared to Trump, none of the Democrats attracted as much toxicity, with the most common themes including “you are an idiot/liar/disgrace/criminal”, “you are not our president”, “you have no idea/you know nothing”, “you should shut up” and “you can’t stop lying”. Many also included references to Vladimir Putin. Trump’s positive mentions tended to include themes such as “God bless you” and “we love you”.
Patel hopes the methodology can be used to help reduce the misuse of Twitter by drawing attention to problematic content before it generates traction. This could prove particularly useful in countering life-threatening misinformation, he said.
“For instance, our methodology automatically identified and grouped tweets pushing a hoax that the Australian bush fires had been caused by arsonists,” said Patel.
“Additional research is necessary, but with some more development there could be a range of potential applications. This methodology could be used for automated filtering or removal of spam, disinformation and other toxic content. This could be done by assigning quality scores to accounts based on how often they post toxic content or harass users.”
The researchers have set up a website for interested individuals to explore the data, and the project’s code can be found on Github.
Project Blackfin was set up in 2019 at F-Secure’s Helsinki headquarters, and is described as a research programme dedicated to the development of decentralised AI for cyber security. The project researchers also hope to take artificial intelligence (AI) to the next level by challenging the common misconception that AIs should mimic human intelligence.
source computerweekly
Industry: Cyber Security

Latest Jobs
-
- IAM Consultant- OKTA
- Germany
- upto €90,000 plus benefits
-
I am looking for an experienced IAM process Manager to help drive forward a series of IAM implementation for a global Manufacturing business, Ideally you will be skilled with Okta and have knowledge of PAM Solutions, You will be responsible for: Driving the design and continuous improvement of complex IAM solutions in close collaboration with business partners Consult on the optimisation of IAM processes and design proper IT-based solutions to meet availability and quality targets Define technical specifications for SW-development (standards, design patterns, test cases, scenarios) and manage the life cycle of designed solutions Actively scan for relevant innovations and new technologies to identify further potential for improving IAM solutions and processes using OKTA Analyse new features of the regular Okta releases We are looking for someone with strong IAM experience as an Architect, Analyst, Technical Engineer, or similar role in the Identity and Security domain Experience with relevant certifications in development/administration, design and configuration of the Okta IAM platforms Familiar with LCM - joiners, movers, leavers, application federation - SAML, OIDC, SCIM and many other IAM terms Good mix of competences in IAM business process and project management concepts and tools e.g., ServiceNow, Jira, PRINCE2, SCRUM (agile)
-
- SAP Security Consultant
- France
- upto €70,000 plus benefits
-
I am looking for an experienced SAP Security Consultant. The ideal candidate will have a strong understanding of SAP security concepts and be able to apply them to real-world scenarios. ideally you will also have experience with Securitybridge or Onapsis, or a similar SAP security tool. Your responsibilities will include: Reviewing and auditing SAP security settings and controls Identifying and remediating security vulnerabilities Implementing security best practices Educating users on SAP security Experience experience in SAP security Experience with Securitybridge or Onapsis, or a similar SAP security tool would be very advantageous Strong understanding of SAP security concepts Excellent problem-solving and analytical skills Excellent communication and presentation skills Fluent in French & English
-
- Post Grad MSc Cyber security - Junior Cyber Risk Analyst wanted. UK
- United Kingdom
- Entry role
-
The perfect start to your new Cyber Security Career. Post Graduate Cyber Risk Analyst Wanted. Are you are fresh from earning your Cyber Security MSc and eager to start your career in Cyber Security? We are looking for a recent post graduate to join a forward thinking Cyber Security Consultancy for the ideal entry role into Cyber Security. Whilst employed industry experience is not expected, as full training and support will be provided, a history of recent education in Cyber Security / Cyber Risk is essential. We are looking for someone with an inquisitive mind, who is confident to ask the right questions and who isn't afraid to challenge the status quo. Superb communication skills are a must (in person, written and verbal) This is a UK based role that is remote first with monthly travel (1-2 a month) to meet with the team and in time to meet clients. If you aren’t available to travel this isn’t the opportunity. We are unable to provide VISA sponsorship as there will be a requirement to achieve Security clearance If you're adaptable, open to fresh perspectives, and excited to be part of a forward-thinking team and looking for an opportunity to help make a difference in a Cyber consulting role, this opportunity is for you. For more information apply here……
-
- Microsoft Exchange Contractor | London | OUTSIDE IR35 | SC Cleared
- London
- OUTSIDE IR35
-
Microsoft Exchange Contractor | London | OUTSIDE IR35 | SC Cleared • We require someone that has experience of migration exchange from windows server 2012 to 2019. • In depth understand of On-Prem exchange server management and deployment. • Experience migrating On-Prem exchange servers from 2012 upwards. • Secure Email Gateway experience essential Due to the nature of the requirement the individual must be commutable to London 2-3 days a week.