F-Secure’s AI reads mean tweets to fight abuse and trolls
Researchers at F-Secure’s Artificial Intelligence Centre of Excellence and the University of Crete’s Forth-IS Institute have developed a novel method for categorising tweets that they hope will, in future, help platforms such as Twitter clamp down on bad behaviour and deal more effectively with abuse, harassment and other forms of malicious activity.
Working under the auspices of F-Secure’s Project Blackfin, researcher Andy Patel and PhD student Alexandros Kornilakis conducted experiments on replies to US president Donald Trump and other US politicians, including the field of Democratic candidates vying to take Trump on in the November 2020 election.
Patel said that the “torrential downpour” of content on social media gave bad actors cover to spread misinformation, hoaxes, lies, scams and fake news, and the inability of sites to stop this kind of behaviour was creating a marketplace for likes, views, subscribes, reviews, and fake accounts.
In their paper, A New, novel method for clustering tweets, Patel and Kornilakis set out a new method of clustering – the process of using machine learning to group phrases or passages into buckets based on their topic.
They developed a methodology that involved processing captured data, converting tweets into sentence vectors, combining said vectors into meta-embeddings, and then creating node-edge graphs using similarities between calculated meta-embeddings, from which the clusters were then derived.
Patel and Kornilakis’ tested their methodology around multiple events, including the 2019 UK General Election which set a new high-water mark for abusive behaviour online. The bulk of their research, however, centred on over a million replies to tweets sent by Trump, the Democratic Party candidates, and congresswoman Alexandra Ocasio-Cortez.
According to Patel, the online mentions of such politicians present an extreme version of what the average Twitter user may have to deal with. Inevitably, they receive lots of engagement, and it usually skews extremely positive or negative.
Patel and Kornilakis classified the mentions by identifiers such as subject-verb-object and overall sentiment and then used their methodology to build an average sentiment score. Based on these identifiers, posts were classified as positive, negative or toxic.
In the case of the Democrats, the most common negative tweets included terms such as “you are an idiot/moron/liar/traitor”, “you will never be president” and “Trump will win”. More positive themes included “we love you”, “you got this” and “you have my vote”.
The little-fancied Andy Yang received by far the most positive replies, followed by Bernie Sanders and Amy Klobuchar, while Alexandra Ocasio-Cortez and Elizabeth Warren received the most toxic posts.
However, when compared to Trump, none of the Democrats attracted as much toxicity, with the most common themes including “you are an idiot/liar/disgrace/criminal”, “you are not our president”, “you have no idea/you know nothing”, “you should shut up” and “you can’t stop lying”. Many also included references to Vladimir Putin. Trump’s positive mentions tended to include themes such as “God bless you” and “we love you”.
Patel hopes the methodology can be used to help reduce the misuse of Twitter by drawing attention to problematic content before it generates traction. This could prove particularly useful in countering life-threatening misinformation, he said.
“For instance, our methodology automatically identified and grouped tweets pushing a hoax that the Australian bush fires had been caused by arsonists,” said Patel.
“Additional research is necessary, but with some more development there could be a range of potential applications. This methodology could be used for automated filtering or removal of spam, disinformation and other toxic content. This could be done by assigning quality scores to accounts based on how often they post toxic content or harass users.”
The researchers have set up a website for interested individuals to explore the data, and the project’s code can be found on Github.
Project Blackfin was set up in 2019 at F-Secure’s Helsinki headquarters, and is described as a research programme dedicated to the development of decentralised AI for cyber security. The project researchers also hope to take artificial intelligence (AI) to the next level by challenging the common misconception that AIs should mimic human intelligence.
source computerweekly
Industry: Cyber Security
Latest Jobs
-
- Public Sector Cyber Security Sales | UK
- England
- N/A
-
Public Sector Cyber Security Sales | UK UK | Remote / Hybrid A cyber security provider is seeking a Public Sector Sales professional to drive growth across UK government and public sector organisations. Must have current Cyber Security sales experience. Responsibilities Generate new business selling cyber security solutions into UK public sector Build relationships with CIO, CISO and senior technology stakeholders Manage the full sales cycle from opportunity to contract close Develop pipeline across central government, local government and public sector bodies Support bids, tenders and framework opportunities Experience Proven cyber security sales experience in the UK Track record selling into public sector organisations Familiarity with CCS, G Cloud or other government frameworks Strong stakeholder engagement and deal management skills Location UK based Security Requirements Eligible to obtain UK Security Clearance
-
- Security Architect | MoD - Security Cleared. OUTSIDE IR35 | Hampshire
- N/A
- Outside IR35
-
Security Architect | MOD | Security Cleared | Outside IR35 | Hampshire Commutable The successful candidate must be willing to undergo DV Clearance, ideally already holding active clearance. You will produce high and low level security architecture documentation, guiding and validating designs for systems deployed within sensitive environments. The role requires providing specialist security input into solution design, service transition and change initiatives, working closely with engineering, operations, client and third party stakeholders. You must have current hands on architectural experience, including VMware secure platform design and virtualisation architecture, alongside AWS expertise. This is an outside IR35 contract- 6 month rolling. Part of a longer term MoD project
-
- Active Directory | RBA engineer | UK Remote | SC Clearable
- United Kingdom
- N/A
-
Technical Active Directory (AD) and RBA specialist needed to play a key part in complex, enterprise scale Active Directory and access transformation programmes. You will work alongside senior team, helping reshape access models, modernise legacy directory structures and strengthen security posture across secure environments. This is hands on delivery within high impact projects where your work directly improves access control, compliance and operational resilience. Active UK Security Clearance required. This is a remote role with client travel. Implementation of Role Based Access Control across large AD estates Restructuring complex permission models, security groups and delegated access Supporting domain controller upgrades and core directory improvements Applying security hardening standards and remediating audit findings Enhancing authentication, policy and access governance frameworks Troubleshooting and resolving technical AD challenges within live environments Producing robust technical documentation and identifying project risks You must have the following technical experience Enterprise Active Directory administration Role Based Access and permission remediation OU design and governance Group Policy management Security group delegation models DNS and DHCP services Kerberos authentication / NTLM PowerShell scripting and automation Azure AD | Entra ID Hybrid identity environments Identity Governance PAM
-
- Identity and Access Management Consultant (Saviynt & Microsoft Entra) | UK
- United Kingdom
- N/A
-
Role summary Technical IAM consultant delivering identity governance and cloud identity solutions to enterprise clients. What you will do Implement / Configure / Deploy Saviynt IGA / Microsoft Entra solutions: Lead technical workshops, gather requirements and translate into solution designs. Troubleshoot complex issues, support testing and deployments. Produce technical artefacts and configuration guides. Key skills Hands-on Saviynt IGA experience (workflow, connectors, access governance). Strong practical knowledge of Microsoft Entra ID / Azure AD identity and access controls. Understanding of identity protocols (SAML, OAuth, OpenID Connect) and hybrid identity. Experience with APIs / REST for integrations and automation. What we are looking for Proven delivery experience in IAM / IGA projects, preferably in consulting. Confident communicator with client-facing delivery exposure.