United States

Social media is increasingly used for large-scale population predictions, such as estimating community health statistics. However, social media users are not typically a representative sample of the intended population --- a &quot;selection bias&quot;. Within the social sciences, such a bias is typically addressed with restratification techniques, where observations are reweighted according to how under- or over-sampled their socio-demographic groups are. Yet, restratifaction is rarely evaluated for improving prediction.  Across four tasks of predicting U.S. county population health statistics from Twitter, we find standard restratification techniques provide no improvement and often degrade prediction accuracies. The core reasons for this seems to be both shrunken estimates (reduced variance of model predicted values) and sparse estimates of each population&#39;s socio-demographics. We thus develop and evaluate three methods to address these problems: estimator redistribution to account for shrinking, and adaptive binning and informed smoothing to handle sparse socio-demographic estimates. We show that each of these methods significantly outperforms the standard restratification approaches. Combining approaches, we find substantial improvements over non-restratified models, yielding a 53.0% increase in predictive accuracy (R^2) in the case of surveyed life satisfaction, and a 17.8% average increase across all tasks.

ICWSM 2022

Correcting Sociodemographic Selection Biases for Population Prediction from Social Media

population health

selection bias

twitter

social media

**ICWSM-2022 will be held from June 6th – 9th in a hybrid format, both in-person in Atlanta, Georgia and online.**

**The Workshop and Tutorial Sessions are accessible to those that registered separately for these sessions. If you registered to attend but are not able to access the these sessions from the [Schedule tab](https://underline.io/events/291/schedule?day=2022-06-06T04%3A00%3A00.000Z), please email icwsm@underline.io
**

The International AAAI Conference on Web and Social Media (ICWSM) is a forum for researchers from multiple disciplines to come together to share knowledge, discuss ideas, exchange information, and learn about cutting-edge research in diverse fields with the common theme of online social media. This overall theme includes research in new perspectives in social theories, as well as computational algorithms for analyzing social media. ICWSM is a singularly fitting venue for research that blends social science and computational approaches to answer important and challenging questions about human social behavior through social media while advancing computational tools for vast and unstructured data.


ICWSM, now in its sixteenth year, has become one of the premier venues for computational social science, and previous years of ICWSM have featured papers, posters, and demos that draw upon network science, machine learning, computational linguistics, sociology, communication, and political science. The uniqueness of the venue and the quality of submissions have contributed to a rapidly growing conference, and a competitive acceptance rate of approximately 20% for full-length research papers published in the proceedings by the Association for the Advancement of Artificial Intelligence (AAAI).


**ICWSM Code of Conduct**

All persons, organizations and entities that attend AAAI conferences and events are subject to the standards of conduct set forth on the [AAAI Code of Conduct for Events and Conferences](https://aaai.org/Conferences/code-of-conduct.php). AAAI expects all community members to formally endorse this code of conduct, and to actively prevent and discourage any undesired behaviors. Everyone should feel empowered to politely engage when they or others are disrespected, and to raise awareness and understanding of this code of conduct. AAAI event participants asked to stop their unacceptable behavior are expected to comply immediately. Sponsors are also subject to this code of conduct in their participation in AAAI events.


Additionally, participants are encouraged to be courteous when sharing screen captures and photographs of conference events. Seek permission when possible and respect requests to take down images if those featured ask. Concerns around code of conduct or inclusion may be sent to icwsm22@aaai.org. If you have any concerns or items to report, please reach out to the General Chairs: Diyi Yang and Yelena Mejova.<br><br>


A ticket is required to attend the event, please use the below URL to register for the event if you have not already done so:

https://aaaiconf.cventevents.com/event/be8c26b0-a76f-4753-babb-de2bd9d2a619/summary

Virtual-Only:
$200 AAAI Member
$125 AAAI Student Member
$225 Nonmember
$150 Nonmember Student

Virtual Silver: Includes discounted conference registration, plus a one-year online new or renewed membership in AAAI.
$299 Silver Regular
$174 Silver Student

Virtual - Workshop and Tutorials:
$45 Regular
$25 Student


Registration Is Required

The International AAAI Conference on Web and Social Media (ICWSM) is a forum for researchers from multiple disciplines to come together to share knowledge, discuss ideas, exchange information, and learn about cutting-edge research in diverse fields with the common theme of online social media. This overall theme includes research in new perspectives in social theories, as well as computational algorithms for analyzing social media. ICWSM is a singularly fitting venue for research that blends social science and computational approaches to answer important and challenging questions about human social behavior through social media while advancing computational tools for vast and unstructured data.

Social media is increasingly used for large-scale population predictions, such as estimating community health statistics. However, social media users are not typically a representative sample of the intended population --- a "selection bias". Within the social sciences, such a bias is typically addressed with restratification techniques, where observations are reweighted according to how under- or over-sampled their socio-demographic groups are. Yet, restratifaction is rarely evaluated for improving prediction.  Across four tasks of predicting U.S. county population health statistics from Twitter, we find standard restratification techniques provide no improvement and often degrade prediction accuracies. The core reasons for this seems to be both shrunken estimates (reduced variance of model predicted values) and sparse estimates of each population's socio-demographics. We thus develop and evaluate three methods to address these problems: estimator redistribution to account for shrinking, and adaptive binning and informed smoothing to handle sparse socio-demographic estimates. We show that each of these methods significantly outperforms the standard restratification approaches. Combining approaches, we find substantial improvements over non-restratified models, yielding a 53.0% increase in predictive accuracy (R^2) in the case of surveyed life satisfaction, and a 17.8% average increase across all tasks.

technical paper

Computer vision applications like automated face detection are used for a variety of purposes ranging from unlocking smart devices to tracking potential persons of interest for surveillance. Audits of these applications have revealed that they tend to be biased against minority groups which result in unfair and concerning societal and political outcomes. Despite multiple studies over time, these biases have not been mitigated completely and have in fact increased for certain tasks like age prediction. While such systems are audited over benchmark datasets, it becomes necessary to evaluate their robustness for adversarial inputs. In this work, we perform an extensive adversarial audit on multiple systems and datasets, making a number of concerning observations- there has been a drop in accuracy for some tasks on CelebSET dataset since a previous audit. While there still exists a bias in accuracy against individuals from minority groups for multiple datasets, a more worrying observation is that these biases tend to get exorbitantly pronounced with adversarial inputs toward the minority group. We conclude with a discussion on the broader societal impacts in light of these observations and a few suggestions on how to collectively deal with this issue.

Two-Face: Adversarial Audit of Commercial Face Recognition Systems

Much attention has been on the behaviors of computer vision services when describing images of people. Audits have revealed rampant biases that could lead to harm, when services are used by developers and researchers. We focus on temporal auditing, replicating experiments originally conducted three years ago. We document the changes observed over time, relating this to the growing awareness of structural oppression and the need to align technology with social values. While we document some positive changes in the services’ behaviors, such as increased accuracy in the use of gender-related tags overall, we also replicate findings concerning larger error rates for images of Black individuals. In addition, we find cases of increased use of inferential tags (e.g., emotions), which are often sensitive. The analysis underscores the difficulty in following changes in services’ behaviors over time, and the need for more oversight of such services.

Shifting our Awareness, Taking Back Tags: Temporal Changes in Computer Vision Services' Social Behaviors

Link prediction has been widely applied in social network analysis. Despite its importance, link prediction algorithms can be biased by disfavoring the links between individuals in particular demographic groups. In this paper, we study one particular type of bias, namely, the bias in predicting inter-group links (i.e., links across different demographic groups). First, we formalize the definition of bias in link prediction by providing quantitative measurements of accuracy disparity, which measures the difference in prediction accuracy of inter-group and intra-group links. Second, we unveil the existence of bias in six existing state-of-the-art link prediction algorithms through extensive empirical studies over real world datasets. Third, we identify the imbalanced density across intra-group and inter-group links in training graphs as one of the underlying causes of bias in link prediction. Based on the identified cause, fourth, we design a pre-processing bias mitigation method named FairLP to modify the training graph, aiming to balance the distribution of intra-group and inter-group links while preserving the network characteristics of the graph. FairLP is model-agnostic and thus is compatible with any existing link prediction algorithm. Our experimental results on real-world social network graphs demonstrate that FairLP achieves better trade-off between fairness and prediction accuracy than the existing fairness-enhancing link prediction methods.

FairLP: Towards Fair Link Prediction on Social Network Graphs

In this work we study the association between the song lyrics and \textit{mood} associations with the song, and conduct a data driven analysis to compare how lyrics and acoustics each contribute to the understanding of a song's mood. We conduct our study on a data set containing nearly one million songs, with song-mood associations derived from user playlists on a streaming platform. We take advantage of  state-of-the-art natural language processing models based on transformers to learn the association between the lyrics and moods. We find that a pretrained transformer-based model in zero-shot setting -- i.e., out of the box with no further training on our data -- is powerful for capturing song-mood associations. Moreover, we illustrate that training on song-mood associations results in a highly accurate model that predicts these associations for unseen songs. Furthermore, by comparing the prediction of a model using lyrics with one using acoustics, we observe that the relative importance of lyrics in comparison with acoustics depends on the specific mood. Finally, we verify if the models are capturing the same information about lyrics and acoustics as humans by conducting a manual annotation task on a hundred songs where we obtain human judgments of mood-song relevance based on lyrics and acoustics in isolation.

The Contribution of Lyrics and Acoustics to Collaborative Understanding of Mood

Online review systems are the primary means through which many businesses seek to build the brand and spread their messages. Prior research studying the effects of online reviews has been mainly focused on a single numerical cause, e.g., ratings or sentiment scores. We argue that such notions of causes entail three key limitations: they solely consider the effects of single numerical causes and ignore different effects of multiple aspects -- e.g., Food, Service -- embedded in the textual reviews; they assume away the presence of hidden confounders in observational studies, e.g., consumers' personal preferences; and they overlook the indirect effects of numerical causes that can potentially cancel out the effect of textual reviews on business revenue. We thereby propose an alternative perspective to this single-cause-based effect estimation of online reviews: in the presence of hidden confounders, we consider multi-aspect textual reviews, particularly, their total effects on business revenue and direct effects with the numerical cause -- ratings -- being the mediator. We draw on recent advances in machine learning and causal inference to together estimate the hidden confounders and causal effects. We present empirical evaluations using real-world examples to discuss the importance and implications of differentiating the multi-aspect effects in strategizing business operations.

Effects of Multi-Aspect Online Reviews with Unobserved Confounders: Estimation and Implication

Moral dilemmas play an important role in theorizing both about ethical norms and moral psychology. Yet thought experiments borrowed from the philosophical literature often lack the nuances and complexity of real life. We leverage 100,000 threads—the largest collection to date—from Reddit’s r/AmItheAsshole to examine the features of everyday moral dilemmas. Combining topic modeling with evaluation from both expert and crowd-sourced workers, we discover 47 finer-grained, meaningful topics and group them into five meta-categories. We show that most dilemmas combine at least two topics, such as family and money. We also observe that the pattern of topic co-occurrence carries interesting information about the structure of everyday moral concerns: for example, the generation of moral dilemmas from nominally neutral topics, and interaction effects in which final verdicts do not line up with the moral concerns in the original stories in any simple way. Our analysis demonstrates the utility of a fine-grained data-driven approach to online moral dilemmas, and provides a valuable resource for researchers aiming to explore the intersection of practical and theoretical ethics.

Mapping Topics in 100,000 Real-life Moral Dilemmas

How we perceive our surrounding world impacts how we live in and react to it. In this study, we propose LaBel (Latent Beliefs Model), an alternative to topic modeling that uncovers latent semantic dimensions from transformer-based embeddings and enables their representation as generated phrases rather than word lists. We use LaBel to explore the major beliefs that humans have about the world and other prevalent domains, such as education or parenting. Although human beliefs have been explored in previous works, our proposed model helps automate the exploring process to rely less on human experts, saving time and manual efforts, especially when working with large corpus data. Our approach to LaBel uses a novel modification of autoregressive transformers to effectively generate texts conditioning on a vector input format. Differently from topic modeling methods, our generated texts (e.g. “the world is truly in your favor”) are discourse segments rather than word lists, which helps convey semantics in a more natural manner with full context. We evaluate LaBel dimensions using both an intrusion task as well as a classification task of identifying categories of major beliefs in tweets finding greater accuracies than popular topic modeling approaches.

Modeling Latent Dimensions of Human Beliefs

The tremendous growth of social media has been accompanied by highly advanced online social network (OSN) technologies. Such advanced technologies have been heavily utilized by perpetrators as convenient tools for deceiving people in online worlds. Social capital has been discussed as a powerful mechanism to leverage interpersonal relationships in social networks in order for an individual to achieve his/her goal.  The beauty of social capital is the ability to materialize non-monetary, less costly, and non-economic resources into tools to solve social problems. In this paper, we aim to leverage social capital (SC) to minimize online users' vulnerabilities to online deception.  In particular, we propose a Social cApital-based FriEnd Recommendation scheme, called SAFER, that can protect OSN users from phishing attacks. We quantify three dimensions of social capital, namely, structural, cognitive, and relational, based on user features obtained from real datasets and model a user's friending behavior based on their social capital.  In addition, to model a user's behavior upon being attacked by phishing attacker, we developed the so-called SER-SEIR (Susceptible, Exposed, Recovered-Susceptible, Exposed, Infected, and Recovered) model as a variant of the SEIR model.  Via extensive simulation experiments based on two real datasets considering bot-based and human-based attackers performing phishing attacks, we demonstrate the performance of four SC-based friend recommendation schemes with three non-SC-based comparable counterparts in terms of the ratio of detecting attackers and the fraction of users in the states of S, E, I, and R. Based on the performance comparison, we analyze the overall trends of their performance in terms of the extent of resistance against phishing attacks by bot or human attackers.

SAFER: Social Capital-based Friend Recommendation to Defend Against Phishing Attacks

The role played by YouTube's recommendation algorithm in unwittingly promoting misinformation and conspiracy theories is not entirely understood. Yet, this can have dire real-world consequences, especially when pseudoscientific content is promoted to users at critical times, such as the COVID-19 pandemic. In this paper, we set out to characterize and detect pseudoscientific misinformation on YouTube. We collect 6.6K videos related to COVID-19, the Flat Earth theory, as well as the anti-vaccination and anti-mask movements. Using crowdsourcing, we annotate them as pseudoscience, legitimate science, or irrelevant and train a deep learning classifier to detect pseudoscientific videos with an accuracy of 0.79.

We quantify user exposure to this content on various parts of the platform and how this exposure changes based on the user's watch history. We find that YouTube suggests more pseudoscientific content regarding traditional pseudoscientific topics (e.g., flat earth, anti-vaccination) than for emerging ones (like COVID-19). At the same time, these recommendations are more common on the search results page than on a user's homepage or in the recommendation section when actively watching videos. Finally, we shed light on how a user's watch history substantially affects the type of recommended videos.

"It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations

Facebook users interact with algorithms every day. These algorithms can perpetuate harm via incongruent targeted ads, echo chambers, or "rabbit hole" recommendations. Education around the machine learning (ML) behind Facebook (FB) can help users to point out algorithmic bias and harm, and advocate for themselves effectively when things go wrong. One algorithm that FB users interact with regularly is User-Based Collaborative Filtering (UB-CF) which provides the basis for ad recommendation. We contribute a novel research approach for teaching users about a commonly used algorithm in machine learning in real-world context -- an instructive web application using real examples built from the user's own FB data on ad interests. The instruction also prompts users to reflect on their interactions with ML systems, specifically Facebook. In a between-subjects design, we tested both Data Science Novices and Experts on the efficacy of the UB-CF instruction. Taking care to highlight the voices of marginalized users, we use the application as a prompt for surfacing potential harms perpetuated by FB ad recommendations, and qualitatively analyze themes of harm and proposed solutions provided by users themselves. The instruction increased comprehension of UB-CF for both groups, and we show that comprehension is associated with mentioning the mechanisms of the algorithm more in advocacy statements, a crucial component of a successful argument. We provide recommendations for increased algorithmic transparency on social media and for including marginalized voices in the conversation of algorithmic harm that are of interest both to social media researchers and ML educators.

Correcting Sociodemographic Selection Biases for Population Prediction from Social Media

Downloads

Next from ICWSM 2022

Two-Face: Adversarial Audit of Commercial Face Recognition Systems

Similar lecture

Characterizing Retweet Bots: The Case of Black Market Accounts

Correcting Sociodemographic Selection Biases for Population Prediction from Social Media

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICWSM 2022

Two-Face: Adversarial Audit of Commercial Face Recognition Systems

Similar lecture

Characterizing Retweet Bots: The Case of Black Market Accounts

Downloads