• Forecasting R (Virtual) Workshop – the pupil’s perspective

    I saw on the NHS-R Community slack channel that the much sought after Forecasting R Workshop led by Bahman Rostami-Tabar was moving to a virtual environment whilst most of the country was shut due to the COVID pandemic. I had previously read the “Forecasting: Principles and Practice” online textbook from cover to cover and learnt the ‘motions’ of time series forecasting but hadn’t really understood the ‘why’. I had used previously black-box style models (Prophet), and understood how to make it work rather than why it works. I was looking forward to the course cementing some of the fundamentals in theory, and then refine the practice in the software.

    It delivered.

    Looking back through the 12 pages of notes I took to go alongside the excellent content produced by Bahman, we covered a huge amount in 6 short, two hour sessions.

    Before even picking up a dataset however, we talked through some of the key factors behind forecasting – the high level process, when we might want to or when we might not want to do it, and in which context it works best.

    A screenshot of a cell phone

Description automatically generated
    The time series forecasting process – prep data, visualise, specify a model, model estimation, fit model, evaluate, forecast output

    Another key item before diving in that we covered were some of the key ‘glossary’ terms that are used, such as the difference between frequency (the time intervals between points in your prediction) and horizon (how far into the future you are going to make your prediction). For some, this might be bread and butter, but adding structure to the language of forecasting really helped me get it clear in my head.

    We spent a good share of the sessions looking at the theory of forecasting, and intermingled this with sessions in R. We moved through baseline predictions using simple average, naïve, seasonal naïve and drift methods (and their corresponding functions) through to exponential smoothing, auto-regressive integrated moving average (ARIMA) and regression.

    Baseline models, what they do, and how to do it in R

    We also looked at several measures within the forecasting toolbox, such as residual diagnostics and forecasting errors. It was going through these that asking Bahman to pause and explain in a bit more detail some of these concepts that really set an instructor led course in a league above trying to work through a textbook, re-reading the same cryptic passage and still being none the wiser.

    My key takeaway from the learning was that R makes it really easy to have a hands off approach to modelling (without a calculator in sight!), but picking apart some of the automation meant I was able to convey what was happening back to colleagues from a much better informed perspective.

    My mind map summary of my key takeaways

    The virtual environment worked well from a pupil’s perspective, with some caveats. First, I found that reading the pre-session work was vital. Although Bahman was very open to going back over content, the pace at which we had to move was quick due to time constraints, so having a sense of the content beforehand really helped slot things in to place.

    Alongside the reading, it was so important to have the lab scripts prepped and ready to go from day one. The NHSR Community team did a great job of getting the projects set up on RStudio Cloud, which meant that everything was ready to go, but if (like me) you wanted to get it running locally on your machine spending the time making sure your setup was working as expected prior to the first session was vital.

    On the whole, I found the course both hugely enjoyable and informative. I am looking forward to integrating all that I learnt into my role as a demand and capacity analyst, finally feeling like I am coming from a perspective of understanding as to what methods to use and how they work. The instructor-led approach meant that I could finally get to grips with ideas that had been mystifying from just reading a textbook, for which I would like to extend my thanks to Bahman, Tom and Alysia for their work on running the course.

  • How NHS-R Community do The Apprentice…

    By Zoe Turner

    One of the tasks on the Apprentice a number of years ago was for the contestants to put on a corporate event, at no small cost to the people attending I might add. It’s a tale often told because one of the contestants was gluten free and no one had accounted for dietary needs amongst the contestants so the poor ‘gluten free lady’, as she was known, was served a fruit salad.

    The reason I’m apparently going off tangent so early in a blog, is that it struck me that the Apprentice is all about throwing people in at the deep end and seeing how they cope. It’s entertainment but clashes with the premise that these are potential apprentices to a ‘British business magnate’ (as Wikipedia calls him). Contrast this with NHS-R and how I came to be attending the Train the Trainer event at the end of 2019 and then helped to run the first of 3 local courses this January, having only just started learning R around 2 years ago.

    Analysts have many expectations made of them. They have to be technical, able to interpret data and communicate themselves clearly to non-analysts. Very rarely though will an analyst be expected to train others. Some may produce or present some training to support or mentor fellow analysts, and even then my experience has always been on the receiving end. Coupled with the fact that I’ve never really had a burning desire to teach, it was a surprise to find myself on a course on how to deliver the NHS-R ‘Introduction to R’ workshop.

    The reason I did it is that my involvement with NHS-R has led to this natural consequence of training others. I started with attending the course myself, then presented at the conference and facilitated an Introduction Course run by NHS-R but hosted by my Trust. I then didn’t hesitate in agreeing to taking on the training.

    NHS-R Community held their first two-day Train the Trainer event in Birmingham organised through AphA (Association of Professional Healthcare Analysts). I was supported to go on this by my manager, Chris Beeley, who is a huge advocate of R and Shiny. Whilst he himself has run several workshops over the years I, notably, have run zero!

    At the TtT (got to get an acronym in there) I had the opportunity to meet lots of motivated people from around the British Isles who were as keen as I was, not only to learn how to teach R but also to talk about data – that happened quite a bit in the breaks. We had an opportunity to present to each other, and that was useful as I learn especially from watching others. Everyone has their own style and it gets perfected over time but I was hugely impressed by how engaging people were and how quickly they could read about a subject that was new to them (we looked at the RStudio presentation slides https://education.rstudio.com/teach/materials/) and then go on to express clearly what they’d just learned.

    I could go on about what I learned at the course, but the proof of its value is in what I did with it. And so on 17th January, Chris and I held a workshop for 15 people from various organisations; some had travelled as far as London and Portsmouth, such is the commitment to learn R. Chris led the workshop and I did the live coding/support role which took the edge off that ‘first time’ feeling.

    This idea of having at least two people in a workshop is a good one, even when the trainer is very experienced. Chris, for example, is very used to running workshops alone, but inevitably people get stuck or things don’t work as they should and so I did the helping while Chris moved between training and coding. It felt, to me at least, like the burden was shared. It helped to ensure that no-one in the group was allowed to fall behind so far that they just gave up.

    Chris and I had gone through the slides beforehand as he’d not gone on the TtT course, and having not written them himself, I wanted to make sure he’d know what was covered. What reassured me was that, as he presented the course, there wasn’t a part of it that I didn’t understand myself and couldn’t cover if I had to take over at that moment. And so the day wasn’t as nerve-racking as I anticipated, and I had fun – which must have been noticeable to the group, as I had an email commenting on how we are clearly a happy team!

    Whilst I haven’t actually run a workshop, I think the process I’ve gone through to get to this point has certainly built up my confidence to do it. I’ve taken every opportunity NHS-R community has offered, from doing the introduction to presenting at the conference, and so this next thing – to run the training myself – hasn’t been so scary after all. I feel like a happy and well-supported apprentice of NHS-R, and the great thing about NHS-R in this is that everyone can be an apprentice too – you just have to get involved.

    Being open-source, all the slides for the trainer course and the introduction course are available on GitHub:

    Train the Trainer course materials can be found at:

    https://github.com/nhs-r-community/train_the_trainer

    Course materials for the underlying Intro to R course are found at:

    https://github.com/nhs-r-community/intro_r

    Zoë Turner, Senior Information Analyst, Nottinghamshire Healthcare NHS Foundation Trust.

    @Letxuga007 and curator of @DataScienceNotts (formerly known as @AppliedInfoNotts)

  • Forecasting R

    By Bahman Rostrami-Tabar

    Democratising forecasting project

    The initiative sponsored by the International Institute of Forecasters provides cutting-edge training in the use of forecasting with R software in developing countries. There is no doubt that many people in developing countries cannot afford high fees to attend forecasting workshops. The project aims to make such a training accessible to those people and provide up-to-date training on the principles of forecasting and create a network of forecasters to conduct research on forecasting with social impact for less developed countries. The training has been delivered in Tunisia, Iraq, Senegal, Uganda, Nigeria, Turkey, Indonesia so far.

    We expand the “Democratising Forecasting” initiative to deliver four forecasting workshops per year for the National Health Service (NHS), UK, in collaboration with NHS-R community. We give our time for free despite the debate that such organisations may afford to pay such trainings.

    Why forecasting for NHS

    Ensuring life-saving services are delivered to those who need them require far more than money, infrastructure and scientific progress. Accurate modelling and forecasting systems can assist critical decisions that drive policy making, funding, research, and development in the NHS. Decision makers are making decisions every day with or without forecasts. However, they are more robust and well-informed in the light of what could happen in the future, and that is where forecasting becomes crucial. However, implementing forecasting principles to support decision-making process requires significant technical expertise. To that end, we aim to organize a two-day workshop on forecasting to teach participants how to apply principles of accurate forecasting using real data in healthcare.

    We are offering four workshops in 2020 as per below:

    • Monday 24th & Tuesday 25th February 2020 – Nottinghamshire – Workshop fully booked
    • Monday 22nd June 2020 – Huddersfield – Online https://www.eventbrite.co.ukworkshop is fully booked
    • Thursday 13th & Friday 14th August 2020 – Gloucestershire
    • Wednesday 13th & Thursday 14th October 2020 – West Midlands
    • Thursday 12th & Friday 13th November 2020 – Cardiff

    Workshop instructor

    Dr. Bahman Rostami-Tabar is the main instructor for the workshops. Colleagues from other Universities in the UK might join him occasionally. Bahman is a Senior Lecturer (Associate Professor) in management science at Cardiff Business School, Cardiff University, UK. He is interested in the use and the Implication of forecasting in social good areas such as health and humanitarian. He has been involved in various forecasting related projects with NHS Wales, Welsh Ambulance Service Trust and the International Committee of the Red Cross. He is currently working on projects that focus on developing innovative forecasting methodologies and their links to decision making in health and humanitarian operations.

    Who should attend

    This workshop is for you if you are:-

    • A decision maker who wants to use forecasting tools and techniques using R to empower decision making;
    • A data analyst who wants to gain in depth understanding of forecasting process.
    • A Forecaster who wants to learn how to use R software for forecasting purpose.

    What participants will learn in the workshop

    Assuming basic knowledge of statistics, participants will be able to do the following tasks when using forecasting to support decision making process in real World:

    • Determine what to forecast according to a forecasting process framework;
    • Prepare and manipulate data using functions in basic R, tidyverse and lubridate packages in R;
    • Identify systematic patterns using time series toolbox in ggplot2 and forecasting related packages in R such as forecast package;
    • Produce point forecasts and prediction intervals using R functions in forecasting related packages such as forecast and fable and user defined functions;
    • Determine the accuracy of forecasting models using statistical accuracy performance measures for point and prediction intervals;
    • Visualize, export and report result for interpretation and insights using RMarkdown.

    Requirement:

    • Basic knowledge in statistics is assumed, e.g. “I know what normal distribution is”;
    • Basic knowledge of R is assumed, e.g. “I know what data type and data structure is”, “I know how to use a function”;
    • No knowledge of forecasting is assumed;

    Program

    Start: 09:30 a.m.

    End: 04:30 p.m.

    Refreshment breaks:

    • Morning: 11:00 – 11:20
    • Afternoon: 03:00 – 03:20 p.m.

    Lunch: 12:30 p.m. – 01:30 p.m.

    Webinar:

    Title: Essentials to do forecasting using R

    Date: Two weeks before the workshop (TBC)

    Day 1

    1.1.Forecasting and decision making: What is forecasting? How forecasting task is different from other modelling tasks? What is the link between forecasting and decision making, how to identify what to forecast?

    1.2. Data preparation, and manipulation: how to prepare data for the forecasting task, how to clean data? How to manipulate data to extract time series?

    1.3. Time series patterns and decomposition: what could be used in data for forecasting task? how to detect systematic pattern in the data? how to separate non-systematic pattern?

    1.4. Forecaster’s toolbox: How to use time series graphics to identify patterns?  what is a forecasting benchmark? What are the simple forecasting methods that could be used as benchmark? How to generate point forecasts and prediction interval using simple forecasting methods?

    1.5. Forecast accuracy evaluation: How do we know if the forecasting method captures systematic patterns available in data? How to judge whether a forecast is accurate or not? How to evaluate the accuracy of point forecasts and prediction interval? Why do we need to distinguish between fitting and forecast?

    1.6. Exponential smoothing models: What is the exponential smoothing family? what are available models in this family? What is captured by this family? how to generate point forecast and prediction intervals using exponential smoothing models?

    Day 2

    2.1. ARIMA models: This is another important family of forecasting models. What is the ARIMA framework? what are available models in this family? What is captured by this family? how to generate point forecast and prediction intervals using ARIMA models? 

    2.2. Regression: We also look at causal techniques that consider external variables. What is the difference between regression and exponential smoothing and ARIMA? How to build a simple regression model?

    2.3. Special events: In addition to using systematic patterns in time series, we discuss how to include deterministic future special events, such as holidays, festive days, etc in models.

    2.4. Forecasting by aggregation:  How to forecast in a situation where we have a high frequency time series, e.g. daily but we need a low frequency forecast, e.g. monthly?  How do we generate forecast for items with a hierarchical or grouped time series structure?

    2.5. Forecasting by combination:  how to use an ensemble of forecasting approaches? In which conditions ensemble forecasts are better than individual methods?

    2.6. Forecasting for many time series: what is the best approach to forecast many time series? How to classify items for forecasting? How to forecast them? How to report the accuracy?  

    Reference:

    Workshop Booklet:

    • Materials will be provided for the workshop in RMarkdown.

    Books:

    Bahman Rostami-Tabar

    Associate Professor in Management Science, Cardiff University, UK

    www.bahmanrt.com