Strength and limitations of the indicators
Strength
The strength of the PC Monitor is that it builds on well-known frameworks for health care system analysis (such as the structure-process-outcome approach) and primary care research [8, 19]. The identified dimensions, features, and indicators are based on the systematic primary care literature review and supported by consensus among primary care experts. Another strength is that in most countries the majority of indicators can be measured by using existing data sources, such as statistics, scientific literature, and policy documents. Some indicators will need an expert opinion for implementation. Furthermore, due to the applied consensus procedure, the Monitor is intended to be applicable to different configurations of primary care (e.g. the different disciplines involved in the provision of primary care).
Limitations
The selection and prioritization of dimensions, features and indicators were subject to decisions on several levels. Starting with the search strategy for the systematic literature review, the review process of publications, the data extraction from publications, and finally the evaluation of indicators by the involved primary care experts. Every step of the development process was conducted in agreement with the PHAMEU project partners from ten countries, to safeguard the importance, scientific soundness, and feasibility of the resulting PC Monitor. However, the application of the PC Monitor by the PHAMEU project in the 31 participating countries will ultimately show its feasibility.
The PC Monitor is not exhaustive. Only dimensions marked as important in the systematic literature review are included in the Monitor. Nevertheless, even though the systematic literature review indicated health equity as an important primary care dimension (because primary care can be a means to achieve equity), it was excluded as a dimension in the Monitor because of a lack of health equity indicators that are valid, feasible and measurable, and subject to primary care. However, aspects that influence equity in use of primary care services are included in the Monitor. It is recommended that future research should focus on the development of suitable equity indicators for primary care research.
The reliance on existing data sources is both a strength and a limitation. It can be a limitation because it could reduce the comparability of the resulting primary care information. The comparability would be optimal when data from uniform international surveys are used.
Application of the PC Monitor
Application of the PC Monitor can be seen as a first test of evaluating what politicians have been 'advertising' about primary care for a while now. The best test of the PC Monitor is to start data collection, as planned in the PHAMEU project. The PC Monitor will be applied in 31 countries by a network of 10 partners. Partners are responsible for data collection in their own and two or three other countries based on their expertise and affinity. Details of the data collection will be tuned to the local situations and availability of sources. For some indicators data can be found in international databases, such as from the OECD, Eurostat, or the WHO Health for All Database. Another source of information are the regularly updated publications in the series 'Health Systems in Transition' (HiT) published by the European Observatory on Health Systems and Policies. Relevant sources can be found via European organisations and networks in primary care (for instance WONCA, EGPRN, EURACT, and EQuiP. Furthermore country information can be found in the international literature. These relatively easy sources will only partly contribute to the data collection for each country. The remainder needs to be found from national sources. As far as national sources can be accessed electronically and in a known language, data can be collected relatively easy by desk research. Websites of national statistical bureaus, professional associations, health inspectorates, educational institutes and national literature databases may be useful. National experts may be needed to get access to grey literature or papers in a foreign language, to help identify sources of missing information, or to deliver 'consensus information'. It is likely that there will be strong heterogeneity of data sources and data. In some countries high quality data for the indicators may be easily available, while in others quality and availability may be low. The network of partners will need to decide about 'softness' criteria for the collected data. If no hard data (e.g. statistics) are available softer data will be applied. For example, in the absence of written sources it may be decided to include consensus among experts. The general principle is to aim for the best available data. This approach is justified as long as the origin of the data is recorded with the data.
It is very likely that not all countries will be able to provide data for each indicator. However, pinpointing gaps in information will also be a valuable result. It will be important that the indicators are evaluated after the PC Monitor has been implemented. This evaluation will result in a final, improved version of the Monitor to be used for future applications.
Expected impact
Europe-wide application of the PC Monitor is expected to result in up-to-date information on the structure, process and outcome of primary care systems, variation in primary care systems across Europe and knowledge about primary care oriented policy strategies (e.g. related to accessibility or integration). The PC Monitor also offers countries the opportunity to evaluate their primary care system in the context of their policy aims. If the PC Monitor were to be implemented on a structural basis (e.g. every 5 years) it would result in knowledge of trends in primary care.
By creating a basis for routine data collection, the PC Monitor could serve the need of various stakeholder groups for reliable and comparable information. Application of the Monitor will provide European and national decision makers with comprehensive comparisons of primary care policies and models of provision that may enable them to improve the effectiveness of primary care. For the research community, application of the PC Monitor could considerably contribute to the base of evidence and thus advance the state of the art of (primary) health services research. It can also serve future actions in this area, such as health system impact assessments.