1 Introduction

Tabular data, found in relational databases, spreadsheets, data lakes, on the web, and as standalone files, is among the most pervasive forms of data in our digital society (Borisov, V et al., 2024) (Shwartz-Ziv, R & Armon, A, 2022) (Cafarella, M et al., 2008). Organizations across the public and private sectors accumulate it at scale and depend on it to make decisions (Jagadish, H et al., 2014). Tables owe this prevalence to their compact and semantically rich structure that encodes entities, their attributes, and the relationships among them in a form that is both machine-readable and amenable to analysis (Codd, E, 1970). Yet the knowledge that makes this data valuable does not just reside in the cells themselves. A table of measurements does not state how a quantity develops over time, where it concentrates, or how two variables relate. Instead, such knowledge is implicit, surfacing only while relevant data is located, integrated, and methodically subjected to computations (Jagadish, H et al., 2014) (Herzig, J et al., 2021) (Ji, X et al., 2024). What people require to act is therefore not the tabular data itself but the knowledge derived from it.

Deriving this knowledge is demanding (De Bie, T et al., 2022), and not only because of the analysis it requires. Organizations often hold vast collections of tables without a precise account of what they contain or where any particular information resides, which makes locating the relevant data itself a substantial task (Bogatu, A et al., 2020) (Nargesian, F et al., 2019). Both locating the data and analyzing it require skills, effort, and familiarity with the data (Jagadish, H et al., 2014) that those who have a need for that knowledge may lack. This is characteristic of the open setting in organizations, where individuals do not know the location, contents, structure, and coverage of the required data in advance and where time, resource, and skill constraints are practical hurdles to extracting knowledge.

For instance, consider a public-health officer who wants to know how nitrogen dioxide (NO2) pollution varies across a city’s boroughs to inform a future traffic policy. Measurements are collected by the city’s sensor network, yet the officer does not know where in the municipality’s data lake they are stored, how they are organized, and how complete they are. The measurements are tagged with coordinates, so further data on administrative boundaries needs to be located and combined to analyze the pollution per borough.

Analytical knowledge remains effectively locked behind the discovery and analysis needed to surface it. Democratizing access to it could do for the knowledge held in tables what web search does for information held in documents, letting anyone with a need obtain insights without a background in data analysis or programming. Realizing this potential calls for treating the problem end-to-end, spanning the full path from the need a person has to presenting the knowledge in a form that delivers insights.

We refer to the pursuit of making the knowledge held in tabular data accessible to those who require it as Open Tabular Insight Extraction (OpenTI). We understand end-to-end OpenTI as a destination that research is advancing toward and whose foundations this work develops.

Open Tabular Insight Extraction (OpenTI) is the task of satisfying a sufficiently directed insight need by deriving and presenting the analytical knowledge that most fully serves it, over a corpus of tabular data in an open setting where corpus contents are potentially diverse and not known when the need is expressed.

The term Open Tabular Insight Extraction captures the characteristics of the task. It is open in that the need is expressed over a corpus whose contents, schema, and coverage are unknown in advance, requiring exploration of the available data. It concerns tabular data, which enables analysis and shapes how data is discovered and combined. It targets insights because the underlying need is satisfied through analytical knowledge that, interpreted within a person’s contextual understanding, yields an insight. And it is extraction because this knowledge is derived from the discovered data through analytical operations.

Several research communities work toward easing access to the knowledge held in tabular data, each from a different starting point. Database (DB) research has long pursued dataset search to surface relevant tables within large collections (Chapman, A et al., 2020) (Nargesian, F et al., 2019), while semantic parsing of natural language into logical forms such as SQL (Liu, X et al., 2025) sits at the intersection of DB and natural language processing (NLP). NLP has further focused on answering questions over tables and reasoning about their structure (Zhou, W et al., 2026) (Lu, W et al., 2025). Information retrieval (IR) has advanced the retrieval of tables relevant to a query (Wang, F et al., 2021) (Chen, P et al., 2024) (Chen, P et al., 2025), building on the open-domain tradition of retrieving evidence before answering (Chen, W et al., 2021) (Herzig, J et al., 2021). Recent research on agents assembles foundation models into systems that plan and execute analytical work over data (Chen, K et al., 2025) (Tang, Z et al., 2025). Human-computer interaction (HCI) has studied visual presentations of analytical results and the interfaces for specifying and refining the analysis (Srinivasan, A & Stasko, J, 2017) (Setlur, V et al., 2016). Each direction advances a complementary facet of OpenTI, yet none spans the whole.

This activity is broad and accelerating faster than surrounding fields. We identify 160 relevant works from 18 venues spanning ML, IR, HCI, DB, and NLP in a systematic literature review, as detailed in Appendix 8.3. Between 2021 and 2025 the number of works relevant to OpenTI rose roughly eightfold, far outpacing the overall growth at the same venues (Figure 1a). Simultaneously, relevant works have increasingly appeared at cross-cutting machine learning venues and specialized workshops, indicating that interest is not limited to any single originating community. OpenTI is thus emerging as an area in its own right.

That this is happening now reflects that the capabilities needed to holistically address the problem are maturing. Advances in retrieval and learned representations of tables enable uncovering relevant data from large corpora (Wang, F et al., 2021) (Chen, P et al., 2024) (Herzig, J et al., 2020), while Large Language Models and the agentic methods built upon them provide flexible means to address the full path from the expression of a need to presenting an insight (Chen, K et al., 2025) (Tang, Z et al., 2025). Capabilities traditionally pursued in isolation can thus be assembled into systems that attempt insight extraction end-to-end. Simultaneously, the range of data analysis workloads that systems are expected to cover is broadening. Figure 1b exemplifies this, showing a clear trend of the scope of benchmarks broadening over time, and only recently expanding beyond lookup and aggregation workloads.

The problem is thus outgrowing the narrow framings under which its facets have been studied. The components and the demand exist, yet the work remains scattered across communities without a shared account of the problem they collectively address.

While the systems proposed across these communities have grown more capable, they remain largely optimized for a narrow target, the information directly queried for, with success measured by how accurately an exact answer is returned. This only partly captures the requirements on surfacing knowledge from tabular data. Information extracted from tables is seldom wanted for its own sake but for the insight it enables for the one who sought it. What is requested and the insight actually sought need not align. The public-health officer from above may ask for a simple average of NO2 per borough, yet that figure reflects where sensors sit rather than where people are exposed, misleading the very policy decision the officer needs to inform.

Existing efforts to systematize the problem approach parts of it but fall short of providing a holistic account that spans the full end-to-end scope. Systematic efforts have organized works on open-domain question answering (Zhang, Q et al., 2023), retrieval augmented generation (Zhao, S et al., 2024), and LLM-agents (Kapoor, S et al., 2024) (Sumers, T et al., 2024), establishing a structured perspective on systems that perform complex tasks over unseen information, yet none address tabular data. Those that do remain largely confined to surveying works in a narrow descriptive manner, organizing works within the conventions of specific research communities under labels such as text-to-SQL, table question answering, or data science agents (Fang, X et al., 2024) (Lu, W et al., 2025) (Liu, X et al., 2025) (Zhou, W et al., 2026) (Tang, Z et al., 2025) (Tian, J et al., 2026) (Chen, K et al., 2025), which we compare against in detail in Appendix 8.2. We argue that these labels reflect facets of one problem, that treating them separately obscures what has been established and what remains open, and that a unified account of that problem is achievable from first principles. This separation is reflected in how works engage with one another, where works cite others that use the same label at six times the rate they cite across labels (Section 2.3). We therefore develop OpenTI as a field of its own, providing the shared vocabulary and problem definition needed to transfer knowledge between communities and to compare systems and benchmarks beyond the conventions of the community each originates from. Overall, we make the following contributions:

1. Conceptual foundation. We establish principled foundations for OpenTI by formalizing it as a unified problem setting that addresses insight needs with analytical knowledge derived through realizations over corpora of tabular data (Section 2). This formulation allows deriving the functional capabilities systems must exercise (Section 3), and it separates establishing that a realization is valid from measuring how well it serves the need, revealing that current benchmarks approximate the former while leaving the latter largely unmeasured (Section 5). We use this framework to position systems and benchmarks from different research communities within a single scheme (Tables 1 and 2).
2. Review and analysis. We systematically review how works from different research areas work towards OpenTI. We apply the conceptual framework to position systems by the functional capabilities they exercise and the way that they orchestrate the interplay between them (Section 3) and analyze benchmarks against desired criteria for robust evaluations (Section 5), showing that current systems and benchmarks only partially cover the scope and requirements of OpenTI.
3. Synthesis. We consolidate fragmented work on the human-side dimensions of OpenTI under a framing of cooperative interaction. We systematize interaction modes and means and examine interpretability, personalization, and contextualization as means of aligning system realizations with users’ latent insight needs (Section 4).
4. Open challenges and research agenda. We derive a research agenda for OpenTI in which directions are anchored in the gaps and limitations surfaced in the review, structured along key questions towards reliable and accessible insight extraction (Section 6).

We find a consistent pattern across existing research. Effort concentrates on performing limited types of analysis over data that is already at hand, while complementary aspects within the end-to-end scope remain thin. 41 of the 58 systems we review lack any capability to retrieve tables, while outputs are predominantly text and rarely contextualized for specific insights. Similarly, we find current benchmarks to be largely unfit for evaluations under an open setting. We analyze inputs from benchmarks and find that they routinely reference columns, files, and values that a user in an open setting could not know, and that they can be interpreted in multiple ways in the majority of instances of all but two benchmarks while largely assessing the validity of outputs or the analysis against a single reference.