United States

Semantic Role Labeling (SRL) has long been referred to as the task of detecting who did what to whom, where, when, and how? Yet most research efforts looked for answers to these questions only at pre-defined syntactical positions in the sentence, disregarding what language users typically understand from discourse and wider document contexts. These implicit arguments are a naturally occurring phenomenon that can be widely beneficial for many natural language understanding challenges. In this work, we propose a method for crowdsourcing implicit semantic arguments on a large scale. To further explore the conceptualization of such arguments by crowd workers, we let them annotate questions and clauses for a given predicate and all its roles. We hope that this dataset will help overcome the data bottleneck for implicit SRL and will allow machines to perform frame semantics on the discourse level.

NAACL 2022

Looking Beyond Syntax: Detecting Implicit Semantic Arguments

predicate-argument relations

implicit arguments

semantic role labeling

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Despite significant progress in Vision-and-Language Navigation (VLN) over the years, existing agents are evaluated on fine-grained instructions that are neither practical nor efficient. In this study, we consider underspecification as a ubiquitous and important phenomenon to bridge the gap between curated instructions and natural human instructions in reality. We introduce a new setting, namely Underspecified vision-and-Language Navigation (ULN). We verify that our instructions are more practical and efficient from the users' perspective. We propose a goal-aware agent with an Exploitation-to-Exploration (E2E) module that estimates step-wise uncertainty. Experimental results show that existing VLN models are still brittle to underspecification. Empirically, our method outperforms the baseline by a large margin in success rate while maintaining a comparable performance on R2R.

ULN: Towards Underspecified Vision-and-Language Navigation

Some of the information that is conveyed in text is not explicitly stated, but instead, is inferred by the reader. While current state-of-the-art language models are very good at recovering explicit information, they still struggle with inferring implicit information from text, and as a result, that information is lost and NLU quality suffers. One such implicit information phenomenon is gapping, a linguistic phenomenon in coordination structures containing VPs, in which a VP that appears in the initial clause is elided from a non-initial clause and can be inferred from the context of the sentence.

Bridging the Gap: Recovering Elided VPs in Coordination Structures

To tackle the rising phenomenon of hate speech, efforts have been made towards data curation and analysis. When it comes to analysis of bias, previous work has focused predominantly on race. In our work, we further investigate bias in hate speech datasets along racial, gender and intersectional axes. We identify strong bias against African American English (AAE), masculine and AAE+Masculine tweets, which are annotated as disproportionately more hateful and offensive than from other demographics. We provide evidence that BERT-based models propagate this bias and show that balancing the training data for these protected attributes can lead to fairer models with regards to gender, but not race.

Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes


The representations in large language models contain multiple types of gender information. We focus on two types of such signals in English texts: factual gender information, which is a grammatical or semantic property, and gender bias, which is the correlation between a word and specific gender. We can disentangle
the model’s embeddings and identify components encoding both types of information with probing. We aim to diminish the stereotypical bias in the representations while preserving the factual gender signal. Our filtering method
shows that it is possible to decrease the bias of gender-neutral profession names without significant deterioration of language modeling capabilities. The findings can be applied to language generation to mitigate reliance on stereotypes while preserving gender agreement in coreferences

Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information

We investigate whether it’s feasible to de-gender resumes to mitigate potential gender bias in algorithmic resume screening. Using a corpus of 709k resumes from IT firms, we first train a series of models to classify the self­ reported gender of the applicant, thereby measuring the extent and nature of gendered information encoded in resumes. We then conduct a series of gender obfuscation experiments, where we iteratively remove gendered information from resumes while preserving the job­-relevant content. Finally, we investigate the trade­off between gender obfuscation and the performance of a resume screening algorithm. We find three main results: (1) there is a significant amount of gendered information in resumes. (2) lexicon-­based gender obfuscation method can reduce the amount of gendered information to a large extent. However, after a certain point, the performance of the resume screening algorithm starts suffering. (3) general-­purpose gender debiasing methods for NLP models such as removing gender subspace from embeddings are not effective in obfuscating gender.


Gendered Language in Resumes and its Implications for Algorithmic Bias in Hiring

Detecting and mitigating harmful biases in modern language models are widely recognized as crucial, open problems. In this paper, we take a step back and investigate how language models come to be biased in the first place. We use a relatively small language model, using the LSTM architecture trained on an English Wikipedia corpus. With full access to the data and to the model parameters as they change during every step while training, we can map in detail how the representation of gender develops, what patterns in the dataset drive this, and how the model's internal state relates to the bias in a downstream task (semantic textual similarity). We find that the representation of gender is dynamic and identify different phases during training. Furthermore, we show that gender information is represented increasingly locally in the input embeddings of the model and that, as a consequence, debiasing these can be effective in reducing the downstream bias. Monitoring the training dynamics, allows us to detect an asymmetry in how the female and male gender are represented in the input embeddings. This is important, as it may cause naive mitigation strategies to introduce new undesirable biases. We discuss the relevance of the findings for mitigation strategies more generally and the prospects of generalizing our methods to larger language models, the Transformer architecture, other languages and other undesirable biases.

The Birth of Bias: A case study on the evolution of gender bias in an English language model

Researchers have devised numerous ways to quantify social biases vested in pretrained language models. As some language models are capable of generating coherent completions given a set of textual prompts, several prompting datasets have been proposed to measure biases between social groups---posing language generation as a way of identifying biases. In this opinion paper, we analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We find out that the practice of measuring biases through text completion is prone to yielding contradicting results under different experiment settings. We additionally provide recommendations for reporting biases in open-ended language generation for a more complete outlook of biases exhibited by a given language model.

Challenges in Measuring Bias via Open-Ended Language Generation

Numerous works have analyzed biases in vision and pre-trained language models individually - however, less attention has been paid to how these biases interact in multimodal settings. This work extends text-based bias analysis methods to investigate multimodal language models, and analyzes intra- and intermodality
associations and biases learned by these models. Specifically, we demonstrate
that VL-BERT (Su et al., 2020) exhibits gender biases, often preferring to reinforce a
stereotype over faithfully describing the visual scene. We demonstrate these findings on a controlled case-study and extend them for a larger set of stereotypically gendered entities.

Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models

Although approximately 50\% of medical school graduates today are women, female physicians tend to be underrepresented in senior positions, make less money than their male counterparts and receive fewer promotions. There is a growing body of literature demonstrating gender bias in various forms of evaluation in medicine, but this work was mainly conducted by looking for specific words using fixed dictionaries such as LIWC and focused on recommendation letters. We use a dataset of written and quantitative assessments of medical student performance on individual shifts of work, collected across multiple institutions, to investigate the extent to which gender bias exists in a day-to-day context for medical students. We investigate differences in the narrative comments given to male and female students by both male or female faculty assessors, using a fine-tuned BERT model. This allows us to examine whether groups are written about in systematically different ways, without relying on hand-crafted wordlists or topic models. We compare these results to results from the traditional LIWC method and find that, although we find no evidence of group-level gender bias in this dataset, terms related to family and children are used more in feedback given to women.

Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback

Due to the complexity of bias and the opaque nature of current neural approaches, there is a rising interest in auditing language technologies. In this work, we contribute to such a line of inquiry by exploring the emergence of gender bias in Speech Translation (ST). As a new perspective, rather than focusing on the final
systems only, we examine their evolution over the course of training. In this way, we are able to account for different variables related to the learning dynamics of gender translation, and investigate when and how gender divides emerge in ST. Accordingly, for three language pairs (en–> es, fr, it) we compare how ST systems behave for masculine and feminine translation at several levels of granularity. We find that masculine and feminine curves are dissimilar, with the feminine one being characterized by a more erratic behaviour and late improvements over the course of training. Also, depending on the considered phenomena, their learning trends can be either antiphase or parallel. Overall, we show how such a progressive analysis can
inform on the reliability and time-wise acquisition of gender, which is concealed by static evaluations and standard metrics

Next from NAACL 2022

ULN: Towards Underspecified Vision-and-Language Navigation

Similar lecture

Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES