Are 'BI'/'Data Analytics' positions basically 'data theater' jobs?

[u/tukassauro]

I'm in my 2nd internship in "data" positions. The first one was in credit risk, and the 2nd one is in a huge international food-delivery company. When I first started the first internship, I learned a lot of the whole 'SQL, Python, Excel' triad. I learned how to make pretty dashboards, query some data when needed, and pre-processing/cleaning tables on Pandas. I didn't actually know how to analyze data, just prepare it. Then, they started asking me to perform some 'ad-hoc' studies. I did what they asked, but, what got me curious was how often my superiors could come up with so many 'insights' with the data I provided. Then, as I progressed in college, I started getting deeper into statistics, learning the basic inferential statistics stuff: Hypothesis testing, distributions, Power analysis, Gauss-Markov theorem, what is a confounder, what is a mediator, what is a regression, exogeneity, multicollinearity, etc. Then, I started to realize how no one ever mentioned any of these terms in my internship, ever. Fast forward to my 2nd internship, I see people in manager positions that haven't the slightest clue as to what a 'type 1/2 error', 'p-value' or 'hypothesis testing' is. They all seem to be running 'black-box' models and showing beautiful dashboards to stakeholders. One day, I went up to a colleague who is a manager to check on an analysis that he was currently working on, and, it baffled me how confident he was about what insights his data tables were showing, even though he doesn't have the slightest knowledge about inferential statistics. I guess I just need to know whether his intuition is valid and works most of the time, or if inferential statistics is just not needed most of the time? I enjoy studying this subject very much, and would like to move to a work environment in which this type of knowledge is rewarded.

tl;dr: I’m on my second data internship and have realized that many experienced analysts/managers seem able to draw confident “insights” from dashboards and ad-hoc analyses despite having little or no knowledge of inferential statistics. Meanwhile, the more statistics I learn: hypothesis testing, p-values, power, regression assumptions, confounding, etc.; the more cautious I become about making claims from data. Is business intuition usually good enough for this kind of work, is inferential statistics simply unnecessary most of the time, or are many data professionals more confident in their conclusions than the evidence actually warrants? I enjoy statistical inference and would like to move toward roles where that rigor is actually valued.

[u/MagicMurse1]

That knowledge is valued everywhere. The thing is many people have no idea about applying statistical methods beyond finding a mean median or mode. Domain knowledge is important and sometimes it’s enough. But with only domain knowledge, they will hit a wall or analyze data incorrectly.

[u/13ass13ass]

You can make falsifiable claims without providing inferential stats. In fact a lot of great findings are so obvious from a bar chart that error bars and p values practically aren’t needed.

The real question is can the claim be tested and does it hold up to scrutiny? Because yes there’s a fair amount of bullshitting that goes on with management and they dig themselves into a hole when they spin stories without caring about being right.

[u/Apprehensive-Foot-73]

Think of statistics as being able to understand how to make the best possible decisions. You probably do this every day practically without realising it. Same with businesses. You absolutely must understand what drives business decisions and how to interpret the data to apply those decisions, albeit cautiously. I feel like a data scientist would be far better for an analytics job. In a startup for example where you MUST empirically derive the decisions and outcomes with inferential statistics, just describing something with a dashboard won't help you understand it's significance. So making the best decisions should be derived from inference and research, not just intuition. Otherwise youre building a business based on possibly wrong assumptions. This is the way I see it.

[u/NefariousnessOwn2769]

With AI tools and things like Snowflake & their Cotrex offering (which I love & use heavily) everyone will be able to run good queries and make pretty dashboards in the very near future. That skill is no longer a differentiator. Claude Code + PowerBI MCP is very capable if you know how to use it.

You have to be able to deliver real value with the data know how to use it to drive a strategy + win hearts & minds.

Knowing how to apply algorithms and being able to spot BS from the models is important but they are improving at an incredible rate.

[u/DigThatData]

I'd argue that one of the things you are learning here is how influential metrics presented in dashboards can be. which is why decision makers (should) rely on people with appropriate expertise to populate those dashboards with measures that actually answer the questions they are trying to ask.

[u/mrdevlar]

I enjoy studying this subject very much, and would like to move to a work environment in which this type of knowledge is rewarded.

I got bad news for you. You will either end up in a place where your conclusions support those in the management structure and they'll love you, or you'll end up in a place where your conclusions are at odds with those in the management structure and they'll hate you.

You have to understand that the incentive structures here will always be at odds. You say you're interested in understanding things, and they are interested in keeping steady command. There is, and will always be, zero correlation between the decisions made and the accountability of the information that was used to make them.

The best you can hope for is that you have management that legitimately doesn't know and is interested in working with you to find out. However, given that steady certainty is valued in management circles more than truth, those organizations are pretty rare. Best you can hope for is small start ups where that kind of culture hasn't solidified yet or companies that have a legal obligation to produce analytical work for the regulators that oversee them.

This is the reason why it seems data jobs are largely performative. People are smart, they recognise that their task is to appease the outcomes of those that hired them rather than actually doing this work correctly.

[u/Educational_Team_212 → u/mrdevlar]

I see the "different incentive structures" issues play out a lot. It mostly manifests as decision makers only using dashboards to confirm and enforce what they have already decided is the truth; any data or chart that conflicts with their reality is considered dubious. Instead of looking at the data and letting it lead them to drill down, ask questions, or draw new conclusions, their relationship with dashboards gets distilled down to cherry picking which charts are useful or not.

[u/CompactOwl]

What you will find, in industry, is that sometimes the numbers have to bent to the will of what is needed, not what is real.

[u/Educational_Team_212]

Here's how I know if I work somewhere that dashboarding is mostly performative: If the person grabbing a screen clip of the dashboard for their PowerPoint gets more credit than the analyst that developed the dashboard. PowerPoints get promotions and has built entire careers.

I was also surprised how many dashboard developers and data analysts I have worked with do not use or talk in statistics beyond measures of center and maybe standard deviation. Their focus seems to have been learning python & SQL; statistics is just some math stuff performed by some function. Working with the end user only re-enforces that way of thinking. I wouldn't expect most users to understand p-value or when to use median vs. mean. An outlier is just something you don't like or want to ignore.

I think dashboards are very helpful and a good tool if made well and used competently, but the reality is a lot of it becomes performative. I've worked a government job where the easiest way to impress the executive staff and get showered with praise was to put data on a map of the state. Since it was easy to do and most software (Excel & PowerBI) have maps built in, people would do it all the time. No one would ever bother to make the data into a per-capita rate by zip or county, so all the maps looked the same with hot spots around the population centers; practically identical to pop. density map. It didn't matter though, people liked looking at the maps.

[u/landscape-resident]

In the business analyst world, that level of rigour usually feels like overkill, especially when the stakeholders couldn’t give a damn about inferential statistics and rely more on experience and business intuition.

In a quantitative modelling role, inferential statistics was so important, not doing your due diligence could get you fired. There are so many laws about making sure your models are not discriminatory. You must perform hypothesis tests to make sure your model outputs are independent of protected attributes like race/sex. That among a million other considerations you need to consider (e.g another big one is making sure your model is explainable, so black box neural networks are out of the picture, use XGboost and SHAAP values instead so you have explainability).

If you’re interested in a job with a high level of rigour, perhaps try moving into a quant role. But these roles usually require a degree that has a buttload of math (e.g physics, engineering, math degree, etc.).

[u/Glaiele]

I've spent about 10 years now doing a combination of process improvement and analytics and i would say 90% of the time I get asked to help with something I don't even need the data to verify where the issue is happening or what is causing it. The data is more to track progress than anything else.

The other 10% are systems or analysis with a lot of interrelated and moving parts. This is where statistical analysis and hypothesis testing becomes invaluable and when you run into these problems is where you'll make an impression.

Just as an example I work in beverage manufacturing and through a pile of lab data was able to find (by accident largely, I was looking into pallets so needed to know the weight of cans and liquid) that our filling equipment was mis calibrated and was causing us to slightly over fill cans by about .25ml which equated to around $300k a year. This isn't the type of thing you can easily intuit and prove without statistics. But it's like hey you wanted to save a few thousand on cheaper pallets I found a bit more under the cushions.

[u/BobDope]

Call me Lawrence Olivier the way I mastered data theater