At Gysho and in my personal role as an advisor, we regularly get involved with start-ups and scale-ups that want to grow faster. Most of these businesses deal with similar questions, like:
- What strategy should we apply to position ourselves for success?
- Which activities should we prioritise to ensure we have the right impact?
- How do we make the best of our limited resources to get things done?
Assuming businesses aspire to do more than act on gutfeel and instinct, we broadly see two main approaches appearing: using traditional methods to gather data and define a strategy, and using ChatGPT as a shortcut to validate decisions.
This blog compares both methods to our own MarqtAI, which we created as an answer to better serve SME’s in their search for growth.
1. Primary Research
Benefits
- Tailored Insights: Primary research provides specific data relevant to the SME's unique market needs.
- Direct Feedback: Engaging directly with customers allows for immediate and actionable insights into preferences and behaviours.
- Competitive Advantage: Unique data can help SMEs differentiate themselves from competitors by understanding niche markets.
Challenges
- Cost and Time Intensive: Conducting primary research can be expensive and time-consuming, which may strain the limited resources of SMEs.
- Expertise Required: SMEs may lack the necessary skills or experience to design and conduct effective research studies.
- Limited Sample Size: Small sample sizes can lead to less reliable data, making it difficult to generalize findings.
2. Secondary Research
Benefits
- Cost-Effectiveness: Secondary research is usually less expensive since it utilizes existing data sources.
- Time Efficiency: It can be conducted quickly, allowing SMEs to gather insights without extensive time commitments.
- Data Availability: The abundance of available information can be an asset, offering a virtually unlimited view into any market in the world.
Challenges
- Relevance and Accuracy: The data may not be perfectly aligned with the SME's specific needs, leading to potential misinterpretations.
- Outdated Information: Secondary data can be outdated, which may not reflect current market conditions or consumer preferences.
- Data Overload: The abundance of available information can be overwhelming, making it challenging for SMEs to extract relevant insights.
Our Take
In the last project, where we applied traditional methods, we needed more resources to make the most of this approach. We hired a graduate to support us over the course of the project. While our graduate did a great job, the company needed to provide guidance, and it took us three months to complete the study.
In current markets, a 3-month lead time to make decisions on go-to-market strategies no longer suffices. This, along with the resource strain studies place on companies, prompted us to look for alternatives.
1. Data Quality
Model training data is not up to date. At the time of writing GPT-4o models were a few months old. For some answers, it may rely on much older data.
We could only verify sources effectively if we forced AI to search online sources one question at a time and manually review each result.
Data was often not specific enough to cover our requirements for highly specialised markets, making AI revert to generalisations instead.
2. Tackling Hallucinations
General chats gave us generic answers presented as information specific to our markets, with no way of verifying accuracy.
Using chats augmented with online search often resulted in hallucinations as it could not find information and made information up (we look at you, Bard!).
Clear Benefits, Unacceptable Issues
We compared notes, looked at our results and objectively evaluated the viability of using standard AI models for our purposes. Whilst we managed to reach our goals, it needed a lot of iterations and fixing to get there.
So whilst fast, it was flawed and needs in depth expertise. We would even go as far to say that this is a big risk for SME"s, do not accept everything ChatGPT or Bing says, we found discrepancies that can really send your strategy off the rails.
Benefits
- Efficiency and Time Savings: AI chats can draw upon their own knowledge and support data collection and analysis.
- Cost-Effectiveness: Utilizing AI chats can lower the costs associated with traditional market research methods, making it accessible for SMEs.
- Scalability: AI tools can easily adjust to varying research needs without significant additional investment.
Challenges
- Limited Understanding of Nuances: AI may struggle with understanding complex human emotions and cultural contexts, potentially leading to misinterpretations.
- Dependence on Quality Data: The effectiveness of AI-driven insights heavily relies on the quality of input data; poor data can lead to inaccurate conclusions.
- Prone to Hallucinations: We commend models on their ambition to answers questions, but if the data is missing, it pushes models to make things up.
1. Priorities for MarqtAI
Ensure a high-quality level of data and insights.
Minimise cost and lead time to get results.
Simplify market research by embedding expertise into the solution.
2. Research Methods & Data Quality
For our research methods, we chose to create a blend of research points found in primary and secondary research, giving a balanced view of the market and ensuring the breadth of data would be good enough to provide high-quality advice. To keep things easy to understand, we split these into functional areas such as branding, segmentation, trends, etc., which are descriptive of their purpose and contents.
Tackling data quality was a longer journey. We quicky decided we wanted to collect up-to-date and near real-time data from online sources to ensure advice was always in tune with the latest developments. Through a combination of automated data collection exercises, programmatic data influencing, and AI-based analyses, we now collect a vast data set for each report we produce.
After many rounds of testing and iterating our analysis methods and the data we collected, we came to a dataset we were happy with. Moreover, the data sets we collect are so extensive and accurate that they closely match the quality we see in primary research methods, rivalling it as the best source for strategic input.
3. Grounded Advice
Our assistant is trained to perform marketing tasks and will augment a user’s own expertise.
The assistant uses the validated and up-to-date research knowledge produced in the data collection and analysis step.
Our assistant is 100% grounded on data, which means it will not answer questions it does not have the answer to, limiting exposure to hallucinations.
