In the past, there was a limited number of people who were able to understand and execute on the concepts of AI and data science. Now, the world has seen an explosion in demand for these skillsets.
Project management is a critical function in any organization, and it becomes even more important when it comes to AI projects. This is because AI projects are often complex with many stakeholders who have different goals and expectations.
The project management process for AI and data science projects is different than what we are used to seeing in other fields. The key difference is that you need to be flexible with your approach when it comes to deadlines, as they can change frequently due to the scientific nature of the work, which fundamentally involves a lot of uncertainty.
We decided to ask a number of project management experts on their opinions as to what are the best practices for managing data science and AI projects.
Project management methodologies for AI and data science
The first question was about the project management methodology used for AI and data science projects.
Something which can be see is that a small percentage of the participants (17.6%) are not using any methodology. The majority seems to be using AGILE or some derivative of it.
When asked about whether they are following any specific methodologies developed for data science and AI, the majority of the participants said they are not using any methodologies. CRISP-DM and the Team Data Science Process seem to be used by a few, with one respondent saying they are building stories in Kanban backlog.
Potential improvements in project management for AI
When asked about the sufficiency of the current project management methodologies for data science projects, only 29.4% give a confident “Yes” answer. It feels like there is something missing in current approaches. This is potentially due to the fact that software development is somewhat different to data science and AI, so it’s difficult for methodologies developed for one area to carry over to data science without any modifications.
This is also reflected in the fact that 88.2% of the participants said that there can be improvements in the current project management methodologies for AI and data science.
When asked about tools, it looks like the king is JIRA. The second most popular answer is “I am not using any tools”.
Challenges and the future of project management for AI
When asked about the biggest challenge in project management for AI and data science, the participants gave a variety of responses. Some of the reasons cited were
- Data science is esoteric and can’t be properly understood by executives
- Ensuring that timelines are being followed
- Data quality and data strategy
- Data science is experiment-driven and results cannot be guaranteed
- Defining KPIs
It looks like there are a multitude of problems, mostly relating to the communication of data science and the scoping of AI projects. It’s clear that education can provide a clear to fixing any such issues, and it’s one of the reasons the Tesseract Academy has been running events and providing courses for non-technical decision makers.
When asked about the future of project management in AI and data science, some of the respondents gave interesting answers:
- “A hybrid approach using various methodologies such as Scrum that take into account possible scope creep and quickly changing definition of work. Faster review/feedback cycles is a big part in this” by Sebastian Belcher.
- “There’s great need of good project and product management in AI and Data Science as the space is still not mature yet apart from Enterprise businesses. At the moment there are various leads (analytics, data science or data engineering) that are trying to do project management alonside managing the team and developing the practice, which is ok for small organisations, but that doesn’t scale. The leads need to be technical leads and help their teams cope with project work and upskill them in collaboration with a delivery/pm/change function which would be their main role to drive change and delivery.” by Angelo Tzimopoulos
- “Mostly a place where experimentation is being understood in concept of timelines and metrics might not be achieved. A place where failure of an initiative should be considered. Additionally, whenever a new project is being planned data quality should be firstly ensured to avoid losing time, mistakes, pitfalls. Data literacy is important across company to achieve that. Clear goals with clear requirements also play an important role. The heavy math and research behind a new project should also being considered whenever we are talking about project management.”
AI, data science and project management
It’s clear that while the space of project management for data science and AI has evolved, there is still lots of work to be done. In order to successfully implement data science and AI projects, companies need to have the right processes in place, and the stakeholders really need to understand the scope and the deliverables of a data science project.
This is why the Tesseract Academy is committed to educating decision makers on topics such as data science, AI and blockchain. If you have any questions or suggestion, feel free to get in touch. We provide both consulting services, as well as online and in-person workshops on all the aforementioned topics, specialising in decision makers with no technical knowledge. Whether you are a CEO, an entrepreneur or a manager, the Tesseract Academy can help you and your organisation fully understand and implement data science and AI.