All Posts By

James Beresford

Microsoft Lead Gartner 2019 Magic Quadrant for Analytics and BI

By | Data Visualisation | No Comments

Hot BI news is the recently announced 2019 Gartner Magic Quadrant for Analytics and BI and Microsoft was the clear leader, driven in part without doubt by the explosive growth of PowerBI .

2019 Gartner Magic Quadrant BI & Analytics

2019 Gartner Magic Quadrant BI & Analytics

PowerBI is the story

PowerBI is advancing at a rate which its key competitors – Qlik & Tableau – are struggling to match. Further underpinning Microsoft’s lead is its ongoing investment in the underlying Azure Data & Analytics Platforms which give it an edge that competitors just can’t match.

One thing I am frequently hearing from the market is that other platforms are struggling to match PowerBI’s compelling price point – some might cynically say Microsoft are using their deep pockets to undercut everyone else. However in a recent conversation I had with a long standing Cognos customer, once they understood what the product could do – and how much cheaper and faster it would be – it drove them to reconsider their strategy.

I freely admit was initially cynical about Self Service BI a few years ago as it rapidly became transparent that for all the slickness of creating great looking reports the tools were still beholden to a clean set of well managed data. Now modern data platforms are reducing the cost and complexity of providing this data, I am now holding the position that self service BI can really deliver on its value rather than just provide a final polish to a Data Warehouse – as long as it is paired with such a platform.

I continue to maintain my (unpopular in some circles) position that Qlik has nothing unique to offer any more and is doomed to irrelevance unless it innovates or at the very least catches up with its competitors. Tableau remains a solid tool for self service analytics, but the absence of an underlying data platform is going to start hurting it before too long. I would expect it’s longevity to be tied to being acquired by someone suitably huge.

I also note the absence of any serious competition from the other two cloud megavendors. Google offers Data Studio & Amazon has Quicksight – but neither rate a mention. I would watch this space carefully as the pace of innovation by both companies is fierce and Google in particular has strong AI / ML capabilities. Both are also ramping up their own data platform services.

Outside the Big 3

If you are on any of the legacy on premise tools in the 2019 Gartner Magic Quadrant for Analytics and BI such as Cognos/IBM, Oracle or SAS then i’d be seriously considering where you go next. The pace of innovation in the cloud is hard to ignore and users risk lagging behind their competitors if they cling to these.

SAP and Salesforce have a strong story within their own source but have their weakness in using data from outside the native platform. Doing anything in SAP is horribly expensive, and my conversations with Salesforce BI users have not left a very positive impression of the tools’ capabilities.

The remainder either are strong in their niches and / or have minimal presence in Australia (Microstrategy is basically unsupported here as there’s no people doing it) so i’ll not pass comment on them.

Where to from here?

If you have reviewed the 2019 Gartner Magic Quadrant for Analytics and BI and decided you want to know more about PowerBI and the Microsoft Data Platform, we can help.

Full disclosure – FTS Data & AI are a Microsoft Gold Partner so this post is a bit biased. However if you are not using PowerBI and are looking at migrating to a more cost effective platform, want to understand how cloud capabilities are transforming data and analytics – or work for Qlik and want to lure me into a dark alley – please contact us.

FTS Data & AI are Microsoft Gold Partners Data Analytics

Microsoft Gold Partners Data Analytics

 

SSIS Integration Runtime Connectivity Testing

By | Data Platform | No Comments

SSIS Integration Runtime Connectivity Testing is hard as there is no physical Azure VM to log in to as part of the Azure Data Factory (ADF). While behind the scenes there is effectively a VM spun up there is no way to access it.

The scenario our Data Platform team faced was reasonably simple – we needed to connect to a 4D database that sat behind the Storman application that our customer used so that we could extract data for their various workloads. Because 4D is not supported by the Generic ODBC source in Azure Data Factory, we needed to use the 4D ODBC driver. This meant using SSIS to leverage the driver.

The client is well managed in terms of security so the target system can only be accessed within their network. Their Azure network was connected to theirs and properly secured, so part of the setup of the SSIS Integration Runtime in Azure Data Factory is to ensure that it is joined to the virtual network.

Houston, we have a problem

SSIS Azure Data Factory

SSIS Azure Data Factory

However, despite all this – we couldn’t get the ODBC connection to work when deployed. Due to stability issues our first suspect was the driver – after all it frequently crashed Visual Studio 2017 / SSDT and configuration was a pain. Also, initially we couldn’t connect on our dev machines as we weren’t on the clients VPN (easily fixed, fortunately). Then we had the wrong target server (again easily fixed).

Once we got on to ADF of course our debugging options got more limited as we now were having to do SSIS Integration Runtime Connectivity Testing without all the tools available on our desktops . Initially we struggled because the runtime was very slow at sharing its metadata (package paths, connection managers, etc.) so we weren’t initially sure it was even able to work with the driver. Eventually we got enough metadata to start playing with the JSON of the task to configure it. However we continued to get errors in ADF that were’t really helping.

Our breakthrough came when we remembered we could just connect to the more familiar and mature environment of the SSIS catalog that is deployed alongside the runtime. We configured the package correctly, ran it and got a more manageable ODBC error – “Cannot reach Destination server”. A quick ping from our desktops proved the server could be pinged, so as a test we used a simple package with just a script task to ping the server. This worked just fine on our desktop, but when deployed the script task reported failure.

So a quick connectivity test helped pin it down to probable network config issue. Now it’s in the Infrastructure teams hands to ensure everything is configured correctly, but at least we have (for now at least) got SSIS & the ODBC driver off the list of probable causes of the issue. It’s also taught us a few things about SSIS Integration Runtime Connectivity Testing.

D365 FinOps Reporting options

By | Data Visualisation | No Comments

The D365 FinOps Reporting landscape is a bit tricky to navigate. The documentation has not keeping up with the pace of product development. This makes the technical complexities difficult to navigate. In this quick post we provide an overview of what options you have. Dynamics 365 Finance and Operations (aka D365 FinOps) is a powerful ERP that can significantly improve business process efficiency, but for us at FTS Data & AI, we also see that there is a lot of additional value to be obtained from the data it captures.

What are the D365 FinOps Reporting options?

The three key tools within the D365 FinOps Reporting suite are:

  • Financial Reporting (aka: FRW)
  • PowerBI
  • SQL Server Reporting Services or SSRS (aka: Document Reporting Services)

There is also a fourth way – which is to pull the data out (aka BYOD or Bring Your Own Database) and report on it with whatever you like, but that’s for another post.

Financial Reporting

The simplest option – Financial Reporting (previously Management Reporting) – has a designer built into the D365 environment. It’s purpose is to provide Management Financial Reporting – i.e, Balance Sheet, Cash Flow, P&L – with some ability to customise your financial reports to your organisations viewpoint, with some filtering available for operational segments.

However its key limitation is that it is Financial in nature only and geared towards providing fairly static, statutory type reporting. Add to that the design and build of reports is very manual and it quickly reaches its limits. A typical example of such a report is shown below – you can see it is very much geared towards line item summaries.

D365 Finance Report Writer

D365 Finance Report Writer

PowerBI

PowerBI is planned to be the key reporting tool for D365 and may well replace the other options over time. A significant number of the embedded reports are already built in PowerBI. Part of any D365 Subscription is a PowerBI Embedded instance. All of the PowerBI reports in D365 are managed through this. Typically the generic reports do not suit the individual business needs and thus they need customising or extending. This is possible – complex to get started – but simple enough to execute once initiated, for example leveraging our Data Visualisation team who have deep experience doing this. Below is a “Before & After” set of examples to show the difference between out of the box reports and ones that have been enhanced by our team.

D365 FinOps PowerBI Report

D365 FinOps PowerBI Report – Before

 

D365 FinOps PowerBI Report

D365 FinOps PowerBI Report – After

A key restriction is that the embedded reports have to run off of the data in the Entity Store. This is a subset of the data in D365, with predefined and scheduled aggregations. To add content to the Entity Store requires a D365 developer, which adds cost to the report development process.

SQL Server Reporting Services (SSRS)

In every module in D365 there is a substantial set of SSRS reports that are included under the name of “Document Reporting Services“. These are embedded into D365 and present views of data with very simple parameters. You can view the technical details of included reports here and there are at least a thousand of them.

As with the PowerBI option it is possible to customise, brand and extend these reports. In this case the data source is any query that can be written against the Application Object Tree (AOT). Importantly this can show transactional as well as aggregate views. This allows for the production of customer facing content such as invoices.

Summary

So there are three options, each with their quirks:

  • FRW for detailed Financial Reporting
  • PowerBI for Aggregate Analytical Reporting
  • SSRS for highly customised reports

It’s not the clearest or most well documented landscape to navigate, but it is generally possible to produce the content you need to meet your organisations reporting requirements.

PowerBI Partner Showcase is published

By | Data Visualisation | No Comments

We are excited to announce the publication of our first PowerBI Partner Showcase! The Government Contract Analysis Tool is a demonstration of our ability to deliver powerful data visualisation solutions. It’s a great piece of work by our team and a big recognition of their capability.

Power BI Partner Showcase - Government Contract Analysis Tool

Power BI Partner Showcase – Government Contract Analysis Tool

 

Based on Open Data this PowerBI Partner Showcase demonstrates our ability to visualise data to provide insight on Federal Government Spending patterns – check it out on the official PowerBI site. It’s interesting to see how many contracts are awarded in June. This is visible in the bottom chart of the first page. In 2002 a whopping 83% of contract were awarded in June. Surprisingly, not an election year.

The Government Contract Analysis Tool

So what purpose does it serve? Government contracting is a vast industry worth billions of dollars with thousands of contracts awarded every year. Such large data presents difficulties in offering useful analysis, resulting in the commercial trends going unnoticed.

Typical questions are:

  • Who are my major competitors for this service?
  • What procurement method should I tender with?
  • When is the best time to tender for work?
  • Where are the most contracts being awarded within Australia?
  • How long will I need to commit for a contract with this agency?
  • What growth has there been for contract values across agencies over time?
  • How much should I tender for?

To remain competitive and drive revenue growth, decision makers need accurate and real-time answers to the above questions. Unfortunately, direct and constructive analysis is difficult with the vast quantity and low quality of the data provided.

FTS Data & AI took the raw data and crafted an effective analysis tool that provides answers to the critical questions that decision makers demand.

Users can now:

  • Identify who are their major competitors, and how much they have earned in contract revenue, thus helping shape price and value propositions.
  • Determine what procurement method is preferred for each service and agency, reallocating resources and refining tender strategy to better suit the favoured method.
  • See when most contracts are awarded in a financial year, and how much the average value is for that contract, allowing ample time to plan executable strategy.
  • Locate where most contracts are being awarded in Australia, enabling users to lift and shift resources into areas with greater potential for work.
  • Track how long different agencies and services are demanding contracts span, giving users the ability to budget time and associated costs of work.
  • Recognise trends in growth of contract value for each agency and service, empowering decisions around service offerings and prices.
  • Monitor market prices for each agency and service, sliced across numerous fields, informing users of various price pressures and trends.

Using Power BI, FTS Data & AI has transformed a low-quality dataset into a fully-interactive reporting tool capable of providing clear and concise answers to the questions that decision makers are tasked with solving to ensure continued business success.