home › Inheritance and donation › Delphi method of expert assessments. General characteristics of the Delphi method and stages. Approaches to assessing the consistency of expert opinions

Delphi method of expert assessments. General characteristics of the Delphi method and stages. Approaches to assessing the consistency of expert opinions

FEDERAL AGENCY FOR EDUCATION

STATE EDUCATIONAL INSTITUTION OF HIGHER PROFESSIONAL EDUCATION

SUMMARY ON THE DISCIPLINE

"RESEARCH OF SOCIO-ECONOMIC AND POLITICAL PROCESSES"

"Delphi method"

Moscow - 2010

Introduction …………………………………………………………………………2

The study of basic concepts and general theoretical and methodological aspects ………………………………………………………………………………...3

Scope of expert methods …………………………………5

The essence of the method and the stages of use …………………………………………..6

Conclusion ………………………………………………………………….14

List of used literature …………………………………………15

INTRODUCTION

The presented work is devoted to the topic "Delphi Method".

This work is relevant in modern conditions. This is evidenced by the frequent study of this topic.

The relevance of this work is due to the great interest in the topic "Delphi Method" in modern science.

The purpose of the work is a deeper and more substantiated study of the Delphi method with the help of the latest domestic and foreign research on this topic.

As part of achieving this goal, the following tasks were set:

1. To study the concepts, as well as the general theoretical and methodological aspects of the method;

2. Designate the scope of the Delphi method;

3. Reveal the essence of the method and the stages of use;

The sources of information for writing a paper on the topic "Delphi method" were basic educational literature, articles and reviews in specialized and periodicals devoted to this topic, reference literature, and other relevant sources of information.

LEARNING BASIC CONCEPTSII AND GENERAL THEORETICAL AND METHODOLOGICAL ASPECTS

One of the most famous correspondence methods is the Delphi method, developed in the 1950s by the RAND Corporation to study military-strategic and military-technical problems. The authors of this method are O. Helmer, T. Gordon, N. Dolki. The Delphi project was patronized by state authorities and was supposed to become the main form of participation of experts in government decision-making on the widest range of problems, but primarily on military issues. It is no coincidence that the first task assigned to the developers of Delphi was to determine, by the method of expert assessments, the system of optimal targets on the territory of the United States for delivering a nuclear strike and the required number of nuclear charges to implement such a plan from the point of view of the USSR leadership. This method became known to the scientific community only in the mid-60s after the publications in the open press of the works of O. Helmer and T. Gordon, who tried to bring the Delphi method beyond the scope of purely military solutions.

The name of this method comes from the Greek city of Delphi, located at the foot of Mount Parnassus, where the temple of Apollo was located, famous for its oracle, which was addressed with a request to solve certain problems not only by ordinary residents, but also by representatives of the ruling political elites. Such a name, as it were, symbolizes the need for the political leadership of various states and other subjects of the political process to use expert knowledge, which, however, is currently generated in councils not with the help of intuitive forms, but with the use of scientific technologies.

Delphi is a method of expert assessments, in which experts independently answer the proposed questions, presenting their answers on paper. In addition to experts, a special organizational group is involved in Delphi, which develops questionnaires, processes the responses received, and also provides experts with the necessary information.

Delphi method- a multi-stage method that provides for the initial isolation of the experts making their judgments and their further multiple adjustments based on the acquaintance of each expert with the judgments of other experts until the spread of estimates is within the predetermined desired range of estimates variation.

The estimates obtained by means of these methods are static and one-time, as a result of which it becomes necessary to re-apply to experts when making a market share forecast for subsequent periods. In addition, the method of internal and external expert forecasting is characterized by a certain degree of subjectivity.

The reliability of the "Delphi" method is considered high when forecasting for a period of 1 to 3 years, as well as for a more distant period of time. Depending on the purpose of the forecast, from 10 to 150 experts can be involved in obtaining expert estimates.

A qualitative approach allows assessing the specifics of each specific situation. In some cases, a careful examination of the various specific elements that define a situation may be more important than a systematic quantitative assessment. A big disadvantage of this method is the excessive subjectivity of the estimates. Old stereotypes of a foreign society can play a fatal role in decision making. J. Simon assessed this approach as "sporadic, based on selective, uncontrolled perception or ideological and personal predilections."

SCOPE OF EXPERT METHODS

Methods of expert assessments are widely used in forecasting and long-term planning, where there are no sufficiently reliable statistical data on the issue under study, where there are several solutions and it is necessary to choose the most preferable of them. Also, these methods are used in the development of new programs in industries that are strongly influenced by new discoveries in the basic sciences.

When analyzing and forecasting the economic situation, a number of difficulties arise:

The impossibility of accurately predicting the consequences of decisions made;

Non-repeatability and impossibility of experimental verification of the proposed course and results of the solution;

The presence of factors that are beyond the control of the decision maker;

The presence of several possible solutions and the need to choose one of them;

The incompleteness of the initial information, on the basis of which it is necessary to form a problem and make a decision (often the initial information is of a qualitative nature and cannot be quantitatively measured).

The prerequisites for the use of expertise are:

Insufficiency and unreliability of information about the state of certain conditions in which the creation and development of products is carried out;

Stochastic (probabilistic) nature of the information object;

Complexity and novelty of problems.

ESSENCE OF THE METHOD AND STAGES OF USE

The method is applied at the stages of formulating the problem and evaluating various ways to solve it. The Delphi method is one of the tools for choosing and evaluating a solution.

The purpose of the method: obtaining consistent information of a high degree of reliability in the process of anonymous exchange of opinions between members of a group of experts to make a decision.

The essence of the method: the Delphi method is a tool that allows you to take into account the independent opinion of all members of the expert group on the issue under discussion by consistently combining ideas, conclusions and proposals and come to an agreement. The method is based on multiple anonymous group interviews.

The organization of the examination is carried out in several stages:

1. Determination of the goals and objectives of the examination.

2. Choice of examination procedure.

3. Selection and formation of a group of experts.

4. Organization of the examination procedure itself;

5. Information processing.

6. Making a decision based on the results of the examination.

First, the problem is posed - the background is determined, the arguments in favor of its solution are considered, and there is a discussion with all interested parties. The main thing here is to recognize imaginary problems. Therefore, when posing a problem, publicity and discussion are necessary.

After the problem is substantiated, the boundaries of its existence are determined, the set of internal and external factors affecting the problem. For this, the central question is singled out and split into sub-questions. At the same time, they try to limit the field only to those questions, without which it is impossible to get an answer to the central question. Further, the goals and objectives of the implementation of the selected problem are formulated. Thus, the main events, factors, central and secondary issues are selected.

It must be borne in mind that with an increase in detail, the accuracy of the examination increases, BUT the consistency of the opinions of experts decreases.

The organizers of the examination shall choose the procedure for carrying out the examination. There are various approaches to this issue. Can be carried out

Individual or group survey,

Full-time or part-time;

Open or closed.

An individual survey consists of interviewing an expert and allows you to make the most of the abilities and knowledge of each expert.

Group - with this method, experts can exchange opinions, can take into account the missed moment by each of them, adjust their assessment. The disadvantage of group opinion lies in the strong influence of authorities on the opinions of the majority of the participants in the examination, in the difficulty of publicly renouncing their point of view, and the psychological incompatibility of some participants in the examination.

Delphi methods are characterized by the following features:

anonymity of expert opinions;

regulated processing, communication, which is carried out by the analytical group over a number of rounds of the survey, with the results of each round being reported to the experts;

group response, which is obtained using statistical methods and reflects the generalized opinion of the participants in the examination

The Delphi method is the most formal of all expert forecasting methods and is most often used in technological forecasting, the data of which are then used in production planning and product marketing. This is a group method in which a group of experts is asked individually about their assumptions about future events in various areas where new discoveries or improvements are expected.

The survey is conducted anonymously using special questionnaires, i.e. personal contacts of experts and collective discussions are excluded. The answers received are compared by special workers, and the summarized results are again sent to the members of the group. Based on this information, group members, while still remaining anonymous, make further guesses about the future, and this process can be repeated several times (the so-called multi-round polling procedure). Once consensus begins to emerge, the results are used as a prediction.

The Delphi method belongs to the class of quantitative methods of group expert assessments. The survey of experts is conducted in 3-4 rounds, consisting of a series of questionnaires, questions are specified from round to round. To carry out this method, it is also necessary to create an analytical group, which, after each round, performs statistical processing of the information received.

First of all, analysts determine the area of preferred quantitative values of objects.

After such a check, the next round is held. The procedure of expert survey according to the "Delphi" method can be divided into several stages.

STAGE 1. FORMATION OF THE WORKING GROUP

The task of the working group is to organize the expert survey procedure.

STAGE 2. FORMATION OF THE EXPERT GROUP.

In accordance with the Delphi method, the expert group should include 10-15 experts in the field. The competence of experts is determined by questioning, analysis of the level of abstracting (the number of references to the work of this specialist), the use of self-assessment sheets.

STAGE 3. FORMULATION OF QUESTIONS

The wording of the questions should be clear and unambiguously interpreted, assuming unambiguous answers.

STAGE 4. EXAMINATION

The Delphi method involves repeating several steps of the survey.

STAGE 5. SUMMARY OF THE SURVEY

For the first round, the experts are asked questions. Answers should be presented in the form of quantitative estimates to the question posed. The answer must be substantiated by an expert.

The analytical group conducts statistical processing of the information received from all experts. To do this, the average value of the parameter under study, the weighted average value of the parameter under study are calculated, the median is determined as the average member of the general series of numbers received from experts and the confidence area. It is more expedient to calculate the confidence area through the quartile indicator. The value of the quartile is equal to ¼ of the difference between the maximum and minimum scores of the series. The confidence region itself will be equal to the minimum score minus the quartile value, the maximum score plus the quartile value.

Experts must necessarily get acquainted with the results and conclusions of analysts, after which the second (next) round is held. According to the results of the presented calculations, experts can see how their opinion corresponds with the opinion of the entire group of experts. They can change their opinions or leave them the same, but in this case put forward counterarguments in their favor. The principle of anonymity is strictly observed. Thus, 2-3 rounds are held. As a result, we obtain a fairly accurate group estimate.

When using the Delphi method, consider the following:

1. Panels of experts must be stable and their numbers must be kept within reasonable limits.

2. The time between rounds of surveys should be no more than a month.

3. Questions in questionnaires should be carefully thought out and clearly formulated.

4. The number of rounds should be sufficient to provide all participants with the opportunity to get acquainted with the reason for a particular assessment, as well as to criticize these reasons.

5. There should be a systematic selection of experts.

6. It is necessary to have a self-assessment of the competence of experts on the issues under consideration.

7. We need a formula for the consistency of assessments based on self-assessment data.

The Delphi method is applicable in almost any situation that requires forecasting, including when there is not enough information to make a decision.

There are several modifications of the Delphi method, in which the basic principles of organization of expertise have much in common. The differences are related to attempts to improve the method through a more reasonable selection of experts, the introduction of schemes for assessing their competence, improved feedback mechanisms, etc. For the convenience of information processing, all modifications, as a rule, imply the possibility of expressing the answer in the form of a number, a quantitative assessment.

But it has drawbacks - for example, the subjectivity of the opinions of specialists participating in the survey, it does not allow experts' opinions to be confronted in a dispute, and a lot of time is spent on it.

In our country, this method was used to determine the main directions of scientific research in the field of computer technology and predict their characteristics, to assess the prospects for the development of industries. In the latter case, the following tasks can be solved using this method:

Determining the timing of work from the issuance of technical specifications for work to the start of operation of the facility;

Determination of priority directions for the development of enterprises in the industry (according to production technology, the most important economic characteristics - the volume of production, the number of employees, the volume of funds, etc.);

Determining the criteria for assessing the significance of scientific developments, etc. The method called "brainstorming" is fundamentally different from the Delphi method for organizing the work of experts, which is also called the "brainstorming" method, the method of collective generation of ideas. This method involves obtaining a solution as a product of the collective creativity of specialists during a session-session held according to certain rules, and subsequent analysis of its results. Its essence lies in the fact that when substantiating the forecast, two tasks are solved differentially:

Generation of new ideas regarding possible options for the development of the process;

Analysis and evaluation of the ideas put forward.

The parent group, led by the Government's Chief Scientific Adviser, based on 360 recommendations formulated by industry groups, identified 6 cross-sectoral strategic themes:

Communications and computers;

New organisms, genetic products and processes;

Achievements in materials science, engineering and technology;

Improving the efficiency of production processes and

The need to conserve the environment and resources;

Improving the understanding and use of social

factors;

Within these 6 strategic directions, the parent group has identified 27 general priority areas for cooperation between the scientific and industrial community.

The parent group also identified 5 major infrastructure priorities:

The need to support a high level of education and vocational training (special importance is attached to the level of training of school teachers in the field of science and technology, on which the qualifications of the next generation of scientists, engineers and technologists depend);

Further maintenance of a high level of fundamental research (especially in multidisciplinary areas);

Development of a communications infrastructure that will put the UK at the center of information flows;

Support for innovative entrepreneurship (financial institutions and the government should constantly review the policy of long-term financing of small innovative businesses and study the impact of the financial climate on innovative activity);

The need for constant revision of the state policy and legislative framework (primarily in such areas as the protection of intellectual property rights in electronic communications, the development of new genetic organisms, investment in advanced communication infrastructures).

Almost all R&D entities of the country participate in the development of priorities. Priorities are determined “from below” and, as a result, are not “alien” to scientific organizations, which, according to the Office of Science and Technology, facilitates and accelerates the very process of reorienting research.

The Delphi method, as an attempt to anticipate the future through a collective procedure, has a number of shortcomings. These are doubts about the reliability of the results obtained by straight-line aggregation of individual opinions as a sample of a group of experts representing the scientific community, as well as the blurring of goals and results, a high probability of developing a deterministic and passive outlook on the future, as well as direct uncritical copying of foreign experience.

At a lower level of aggregation - regional, sectoral or problematic - in a number of countries, for example in Germany, a study of promising priorities is carried out using the Mini-Delphi method.

So, although the Delphi method is very popular, its influence on the real structure of priorities in most developed countries should still be considered limited. In many countries, this and other methods of identifying priorities often fall on barren ground, that is, they are either not provided with implementation mechanisms or give way to other priorities chosen in accordance with political or any lobbying interests.

CONCLUSION

Summing up, I note that the Delphi method has undoubted advantages over methods based on the usual statistical processing of the results of individual surveys.

It promotes the development of independent thinking of group members, and also provides a calm and objective study of problems that require evaluation. The method allows to reduce fluctuations in the entire set of individual responses, limits fluctuations within groups. At the same time, as the experiments show, the presence of low-skilled experts has a less strong influence on the group assessment than simple averaging of the results of the answers, since the situation helps them correct the answers by obtaining new information from their group.

LIST OF USED LITERATURE

Avdulov P.V., Goizman E.I., Kutuzov V.A. and other Economic-mathematical methods and models for the leader. M.: Economics 2003

Agafonov V.A. Analysis of strategies and development of integrated programs. M.: Nauka, 2005

Beshelev S.D., Gurvich F.G. Mathematical and statistical methods of expert assessments. 2nd ed. revised and additional M: Statistics. 2000-263s.

Bobrovnikov G.N., Klebanov A.I. Forecasting in management

technical level and quality of production: Proc. Allowance.-M: Publishing house

standards. 2001-232s.

L.P. Vladimirova. Forecasting and planning in market conditions., textbook (second edition). M.: 2001

Mathematical methods in planning industries and enterprises / Ed. I.G. Popov. M.: Economics, 2003

Other names of the method: "Delphic method", "Method of the Delphic oracle".

Purpose of the method

It is used at the stages of formulating a problem and evaluating various ways to solve it. The Delphi method is one of the tools for choosing and evaluating a solution.

Purpose of the method

Obtaining agreed information of a high degree of certainty in the process of anonymous exchange of views between members of a group of experts to make a decision.

The essence of the method

The Delphi method is a tool that allows you to take into account the independent opinion of all members of the expert group on the issue under discussion by consistently combining ideas, conclusions and proposals and come to an agreement. The method is based on multiple anonymous group interviews.

Action plan

Form a working group to collect and summarize the opinions of experts.
Form an expert group of specialists who have questions on the topic under discussion.
Prepare a questionnaire, indicating in it the problem posed, clarifying questions. The wording should be clear and unambiguously interpreted, suggest unambiguous answers.
Conduct a survey of experts in accordance with the methodology, which, if necessary, repeats the procedure. The responses received serve as the basis for formulating questions for the next stage.
Summarize expert opinions and issue recommendations on the problem posed.

Method features

Delphi", "Delphic method", "method of the Delphic oracle" come from the name of the place Delphi, where oracles-soothsayers lived at the temple of the god Apollo (Ancient Greece).

The word of the main oracle was taken as the ultimate truth.

It is known that the use of collective knowledge leads to the possibility of finding strong solutions, however, in the process of exchange of opinions between participants, the influence of the influence of the authority of colleagues can affect and it all comes down to the emergence of popular answers.

The Delphi method allows us to resolve this dialectical contradiction. To do this, direct discussions of experts are replaced by individual surveys. The collected answers are subjected to statistical processing. The received generalized answers are transmitted to each expert through personal communication, or by mail or e-mail with a request to review and clarify his opinion, if he deems it necessary. This procedure may be repeated several times.

Examination by the Delphi method

Additional Information:

The Delphi method is a systematic way of summarizing expert assessments.
It is believed that the Delphi method is most applicable if experts are involved in the work who are competent not on the whole problem, but on its various components.
To decide whether to use the Delphi method, it is very important to carefully consider the situation to which the method will be applied. And before making a decision, you need to ask a number of questions:

who will conduct the examination, and where its participants will be located;
what kind of communication should be maintained with them in the process of considering the existing problem;
what alternative methods are available and what results can realistically be expected from their application?

Advantages of the method

The Delphi method contributes to the development of independent thinking of group members.
Provides a calm and objective examination of issues that require evaluation.

Disadvantages of the method

Excessive subjectivity of assessments.
Requires a lot of time and organizational effort.

Expected Result

An agreed list of ideas with their associated strengths and weaknesses.

MINISTRY OF ECONOMIC DEVELOPMENT AND TRADE OF THE RUSSIAN FEDERATION

NIZHNY NOVGOROD COMMERCIAL INSTITUTE

Abstract on the topic:

DELPHI METHOD

Performed:

Student 4-1EF gr.

Maltseva Ya.V.

Checked:

Zhelonkin V.V.

Nizhny Novgorod

Introduction

Scope of expert methods.

When analyzing and forecasting the economic situation, a number of difficulties arise:

The impossibility of accurately predicting the consequences of decisions made;

Non-repeatability and impossibility of experimental verification of the proposed course and results of the solution;

The presence of factors that are beyond the control of the decision maker;

The presence of several possible solutions and the need to choose one of them;

The prerequisites for the use of expertise are:

Insufficiency and unreliability of information about the state of certain conditions in which the creation and development of products is carried out;

Stochastic (probabilistic) nature of the information object;

Complexity and novelty of problems.

The organization of the examination is carried out in several stages:

1. Determination of the goals and objectives of the examination.

2. Choice of examination procedure.

3. Selection and formation of a group of experts.

4. Organization of the examination procedure itself;

5. Information processing.

6. Making a decision based on the results of the examination.

Determination of the goals and objectives of the examination

It must be borne in mind - with an increase in detail - the accuracy of the examination increases, BUT the consistency of the opinions of experts decreases.

The organizers of the examination shall choose the procedure for carrying out the examination. There are various approaches to this issue. Can be carried out

-individual or group survey,

- full-time or part-time;

- open or closed.

Individual survey consists in interviewing an expert and allows you to make the most of the abilities and knowledge of each expert.

Group - With this method, experts can exchange opinions, take into account the missed moment by each of them, and correct their assessment. The disadvantage of group opinion lies in the strong influence of authorities on the opinions of the majority of the participants in the examination, in the difficulty of publicly renouncing their point of view, and the psychological incompatibility of some participants in the examination.

From Methods group surveys are used:

various modifications Delphi method.

Delphi Methods are characterized by the following features:

anonymity of expert opinions;
regulated processing, communication, which is carried out by the analytical group over a number of rounds of the survey, with the results of each round being reported to the experts;
group response, which is obtained using statistical methods and reflects the generalized opinion of the participants in the examination

Delphi method is the most formal of all expert forecasting methods and is most often used in technological forecasting, the data of which are then used in production planning and product marketing. This is a group method in which a group of experts is asked individually about their assumptions about future events in various areas where new discoveries or improvements are expected.

The application of the Delphi method can be illustrated by the following example #1: An offshore oil company wants to know when it will be possible to use robots instead of divers to inspect platforms underwater. To start forecasting using this method, a company must get in touch with a number of experts. These experts should come from a wide variety of industries, including divers, oil company engineers, ship captains, maintenance engineers and robot designers. They explain the company's challenge, and each expert is asked when he thinks it will be possible to replace divers with robots. The first answers will probably give a very wide spread of data, for example, from 2000 to 2050. These responses are processed and returned by experts. At the same time, each expert is asked to revise their assessment in the light of the answers of other experts. After repeating this procedure several times, opinions can converge, so that about 80% of the answers will give a period from 2005 to 2015, which will be sufficient for the purposes of production planning and implementation of robots.

The Delphi method is named after the Delphic oracle in Ancient Greece. It was developed by Olaf Helmer, a prominent mathematician at the RAND Corporation, and his colleagues, which is probably why, compared to other creative approaches, it gives sufficient prediction accuracy.

First of all, analysts determine the area of preferred quantitative values of objects.

After such a check, the next round is held. The procedure of expert survey according to the "Delphi" method can be divided into several stages.

STAGE 1. FORMATION OF THE WORKING GROUP

The task of the working group is to organize the expert survey procedure.

STAGE 2. FORMATION OF THE EXPERT GROUP.

STAGE 3. FORMULATION OF QUESTIONS

The wording of the questions should be clear and unambiguously interpreted, assuming unambiguous answers.

STAGE 4. EXAMINATION

The Delphi method involves repeating several steps of the survey.

STAGE 5. SUMMARY OF THE SURVEY

For the first round, the experts are asked questions. Answers should be presented in the form of quantitative estimates to the question posed. The answer must be substantiated by an expert.

The analytical group conducts statistical processing of the information received from all experts. To do this, the average value of the parameter under study, the weighted average value of the parameter under study are calculated, the median is determined as the average member of the general series of numbers received from experts and the confidence area. It is more expedient to calculate the confidence area through the quartile indicator. The quartile value is equal to ¼ of the difference between the maximum and minimum scores of the series. The confidence region itself will be equal to the minimum score minus the quartile value, the maximum score plus the quartile value.

Example #2: The problem is to estimate the level of demand for good A in 2003. 10 experts are invited. Each expert received a questionnaire with a description of the product and the intended market. The experts were asked to give themselves an individual self-assessment in points in the range from 0 to 10. The level of demand is proposed to be estimated in% (percentage) in the range from 0 to 100.

Each expert works independently and anonymously. After the 1st round, the following results were obtained from the experts:

expert number	self-esteem coefficient	demand level - individual expert assessment

The analysis team makes the following calculation:

The average group self-esteem is = (10 + 8 + ... + 9.9) : 10 = 8.61

The average value of demand (simple estimate) is equal to (90+100+…+80) :10 =83.5%

The weighted average estimate of demand is (10x90 +8x100+…+9.9x80) : (10+8+…+9.9) = 84.1%

The median in this case, with an even number of experts, is calculated as the arithmetic mean between the median estimates and will be equal to Me = (80+80):2=80

The confidence area is calculated as follows:

The minimum score from the set of expertise is determined - 60%;

the maximum score is -100%.

The quartile will be equal to (100-60):4=10%.

Therefore, the lower bound of the confidence region will be equal to 60+10=70%,

the upper limit will be 100-10=90%.

application area

								expert assessment

Rice. Confidence area

All the results obtained are submitted for consideration by the experts. If the experts consider it appropriate to correct their opinion, they pass their corrections to the analytical group. And the analytical group calculates new results according to the algorithm that was discussed above.

The final generalized opinion is the basis for forecasting the level of demand for this product A.

When using the Delphi method, consider the following:

1. Panels of experts must be stable and their numbers must be kept within reasonable limits.

2. The time between rounds of surveys should be no more than a month.

3. Questions in questionnaires should be carefully thought out and clearly formulated.

4. The number of rounds should be sufficient to provide all participants with the opportunity to get acquainted with the reason for a particular assessment, as well as to criticize these reasons.

5. There should be a systematic selection of experts.

6. It is necessary to have a self-assessment of the competence of experts on the issues under consideration.

7. We need a formula for the consistency of assessments based on self-assessment data.

The Delphi method is applicable in almost any situation that requires forecasting, including when there is not enough information to make a decision.

Some disadvantages of the Delphi method are related to the lack of time allotted to the expert to think about the problem. In this case, the expert may agree with the opinion of the majority in order to avoid the need to explain what is the difference between his decision and the other options. These shortcomings are eliminated by improving the organization of examinations by creating automated systems for processing survey results. The technical implementation of such a system is based on the use of computers with external terminals (displays). The computer ensures the presentation of questions to experts (communicating with it through their personal displays), the collection and processing of the results of answers, the request and issuance of arguments and other necessary information for preparing answers.

In addition, some experts believe that "the assumption that those who sharply disagree with the opinion of the majority, justify their point of view, may lead to an increase in the effect of accommodation, and not reduce it, as intended." But still, many scientists argue that the Delphi method is superior to "conventional" forecasting methods, at least when developing short-term forecasts.

The Delphi method was first described in the "Report on the study of long-range forecasting" by the American Rand Corporation in 1964. The objects of the study were: scientific breakthroughs, population growth, automation, space exploration, the emergence and prevention of wars, future weapons systems. Over the past period, the range of predictable processes using the Delphi method has expanded significantly, but there is no doubt that this method has found the greatest application in areas related to scientific and technological progress.

In particular, in our country, this method was used to determine the main directions of scientific research in the field of computer technology and predict their characteristics, to assess the prospects for the development of industries. In the latter case, the following tasks can be solved using this method:

Determining the timing of work from the issuance of technical specifications for work to the start of operation of the facility;

Generation of new ideas regarding possible options for the development of the process;

Analysis and evaluation of the ideas put forward.

Usually, all specialists during the meeting are divided into two groups, consisting of the same or different representatives, so that one group generates ideas, and the second analyzes them. At the same time, during the meeting it is forbidden to express any critical assessment of the value of the idea; as many of them as possible are encouraged, since it is assumed that the probability of a really valuable idea coming up increases with an increase in their total number; free exchange of opinions is encouraged; the expressed thoughts should be picked up and developed, etc. The session is led by an impartial moderator. His task is to direct the development of the discussion in the right direction, towards the achievement of a given goal, without getting lost in conversation, wit competition, etc. At the same time, he should not impose his opinion on the participants in the discussion, orient them to a certain way of thinking.

For Russia, the formulation of goals and the development of methods for selecting the priorities of the state scientific and technological policy is of particular importance. Although the compilation of comprehensive forecasts of the scientific and technological development of the country and the world in the USSR began in the early 1970s, the main guidelines for them were the interests of the defense sector and the party state apparatus. At present, the development goals have certainly expanded, but the procedure for selecting priorities corresponding to them has not been developed, agreed upon, has no regulatory framework and traditions. Under these conditions, when choosing priorities and obtaining appropriate financial and legal support, the displaced and narrow interests of departments, the military-industrial complex, regions, or someone else may prevail, while the interests of the state as a whole will not be taken into account. Under these conditions, the development of the procedure for selecting priorities and the study of the experience of other countries are extremely important.

In most developed countries, the following methods are used to determine the priorities of scientific and technological development in the process of forecasting and making decisions on financing large government programs:

Delphi
Compilation of a list of critical technologies.
Expertise

Technology forecast based on Delphi method, — it is an attempt to predict the development of a particular technology in the long term (20-30 years). Developed for the first time in the 50s by RAND Corp., the Delphi technique was used for the first time for the purposes of national and sectoral technological forecasting by Japan (since 1970, 6 studies have already been completed), and subsequently, and largely following the Japanese model, by Germany, France , Great Britain, Spain, Austria, South Korea mainly during the last decade (we can talk about the boom of this method in the 90s).

The Delphi method consists in the evaluation of technologies by experts, (their number varied from 123 people in Spain, to 25 thousand at the first stage — in South Korea) on the basis of proposed schemes that include several positions, including the level of research activity in this area, participation in the creation of national wealth, improving the quality of life and competitiveness, and the expected timing of the implementation of new achievements. The two-four-stage evaluation procedure allows experts to clarify or revise their point of view, taking into account the opinions of colleagues and, as a result, develop an agreed, truly collective position on the entire range of issues raised, the number of which at the first stage, as a rule, exceeds a thousand.

Forecasting according to the Delphi method is also effective in achieving a number of other results that are fundamentally important for identifying priorities. This is a cognitive effect, training and broadening the horizons of experts participating in the survey, mapping competencies in individual disciplines, technical fields and countries, developing a consensus among representatives of various sectors of the scientific and technical sphere and, last but not least, stimulating a broad discussion by the scientific community of trends in scientific and technological development. their country and the world.

Japan has not only the longest history of predictive estimates of the technological development of its country and the world, but also the most effective practice of using these forecasts for the general orientation of the national scientific and technical sphere, all the more interesting, since the share of the state in financing national science has never exceeded 20-25 %. The Department of Science and Technology, which coordinates the fundamental and applied research of other departments through strategic research programs, is also responsible for technological forecasting.

The Delphi survey is conducted every five years with a time range of up to 30 years, gradually covering all areas of science and technology. If the first survey, which predicted for the period 1970-2000, was able to cover 5 areas and 644 topics, then the last one, covering the period 1996-2025, already included 14 areas and 1072 topics:

materials and their processing;
computer science;
electronics;
life sciences;
health and welfare;
study and use of outer space;
earth sciences and oceanology;
energy and natural resources;
ecology;
agriculture, forestry and fish farming;
industrial production;
urbanization and construction;
connection;
transport.

Respondents in the latest survey were asked to evaluate technology topics in terms of their contribution to socio-economic development, improving the quality of life and solving environmental problems, as well as their importance in general. The survey participants had to determine the time range during which the listed technologies will be implemented both in Japan and other leading countries, as well as outline the range of measures that government authorities need to take for this.

In France, at the beginning of 1994, using the Delphi method, a broad survey of the prospects for the development of 15 major scientific and technical areas (electronics, elementary particle physics, environmental problems, urbanization, etc.) was carried out. More than 1,000 specialists from various sectors of the economy were involved in the expert assessments - 45% representatives of industrial science, 30% of state research institutes and 25% university employees, which generally reflected the structure of the scientific sector of the French economy. The same principle was followed in the formation of expert groups by most of the countries starting work on forecasts and priorities.

In 1991, the German Ministry of Research and Technology conducted a comparative analysis of the assessments of Japanese and German experts using a Japanese questionnaire. The results generally showed the similarity of the positions of experts for the two countries regarding the development of promising technologies, although certain differences were revealed, reflecting the national cultural and industrial specifics of these countries.

In the UK, since 1994, the use of the Delphi method has also begun to select national scientific and technical priorities. However, unlike Germany and France, the country did not follow the path of copying the Japanese experience (for example, in France, when polling expert scientists, the question of the prospects for research on rice growing problems, directly borrowed from the Japanese methodology, was posed as a priority).

The new mechanism for determining the priorities of the state science policy in the UK has been called "Foresight" ("Foresight"). The program will work with industry to identify promising markets and technologies for the next 10-20 years, as well as activities that will take advantage of new opportunities to improve the quality of life and accelerate economic growth. The goals of “Foresight” are: firstly, to collect the information necessary for decision-making on the state and directions of state-funded R&D, secondly, to create a new culture of interaction between scientists and business, and thirdly, to determine the resources necessary to achieve the set goals.

Distinctive features of the new approach are the definition not of specific technologies, but of development directions, the multivariance of scenarios, and the continuity of program stages over time. The Foresight 1 program operated in 1994-1999. and moved to "Foresight II" - 1999-2004. Each program consists of three "mutually flowing" stages - analysis, dissemination of information and application of the results, preparation for the next program. "Foresight" determines the state priorities in scientific and technical programs, in personnel training, in the methods of state regulation. At the same time, Foresight is not a rigid guide for the public sector, but for private industry it serves as an “invitation to action” both in the field of participation in cooperation programs and in the field of strategic planning.

In the first phase, 16 thematic groups, comprising experts from industry and the public sector, analyzed a wide range of markets and technologies. Almost all groups are headed by representatives of large companies and operate in the following areas: agriculture; natural resources and environment; chemical products; means of communication; building; defense and aerospace industries; energy; Financial services; food products; health and life sciences; education and leisure; production processes and entrepreneurship; materials; retail; transport; maritime technologies). Experts using the Delphi method analyzed the points of view of 1000 people. On the basis of these submissions, the groups produced reports assessing future markets and activities needed to maintain the UK's international competitiveness.

The parent group, led by the Government's Chief Scientific Adviser, based on 360 recommendations formulated by industry groups, identified 6 cross-sectoral strategic themes:

— communications and computers;

— new organisms, genetic products and processes;

— advances in materials science, engineering and technology;

— improving the efficiency of production processes and

— the need to conserve the environment and resources;

— improving the understanding and use of social

factors;

Within these 6 strategic directions, the parent group has identified 27 general priority areas for cooperation between the scientific and industrial community.

The parent group also identified 5 major infrastructure priorities:

- the need to support a high level of education and professional training (special importance is attached to the level of training of school teachers in the field of science and technology, on which the qualifications of the next generation of scientists, engineers and technologists depend);

— further maintenance of a high level of fundamental research (especially in multidisciplinary areas);

- development of a communication infrastructure that will allow the UK to be at the center of information flows;

- support for innovative entrepreneurship (financial institutions and the government should constantly review the policy of long-term financing of small innovative businesses and study the impact of the financial climate on innovative activity);

— the need for constant revision of the state policy and legislative framework (primarily in such areas as the protection of intellectual property rights in electronic communications, the development of new genetic organisms, investments in progressive communication infrastructures).

At a lower level of aggregation - regional, sectoral or problematic — in a number of countries, for example in Germany, a study of promising priorities is carried out using the Mini-Delphi method.

Conclusion

The Delphi method has undeniable advantages over methods based on the usual statistical processing of the results of individual surveys. It allows you to reduce fluctuations in the entire set of individual responses, limits fluctuations within groups. At the same time, as the experiments show, the presence of low-skilled experts has a less strong influence on the group assessment than simple averaging of the results of the answers, since the situation helps them correct the answers by obtaining new information from their group.

List of used literature

Avdulov P.V., Goizman E.I., Kutuzov V.A. and other Economic-mathematical methods and models for the leader. M.: Economics 1998
Agafonov V.A. Analysis of strategies and development of integrated programs. M.: Nauka, 1997
Mathematical methods in planning industries and enterprises / Ed. I.G. Popov. M.: Economics, 1997

L.P. Vladimirova. Forecasting and planning in market conditions., textbook (second edition). M.: 2001

An important method in the methodology of management research, as well as system analysis, is the Delphi method. Other names of the method: "Delphi method", "Delphi oracle method". The Delphi method, or "brainstorming method", is a method for quickly finding solutions based on their generation in the process of brainstorming conducted by a group of specialists and selecting the best solution based on expert assessments. The Delphi method is used for expert forecasting by organizing a system for collecting and mathematical processing of expert assessments.

The method is applied at the stages of formulating the problem and evaluating various ways to solve it. The Delphi method is one of the tools for choosing and evaluating a solution; it involves preliminary familiarization of experts with the situation using a model.

The purpose of the method is to obtain consistent information of a high degree of reliability in the process of anonymous exchange of opinions between members of a group of experts to make a decision. The essence of the method is to be a tool that allows you to take into account the independent opinion of all members of the expert group on the issue under discussion by consistently combining ideas, conclusions and proposals, and come to an agreement. The method is based on multiple anonymous group interviews.

The method is based on the idea of improving and saturating information with the help of expert assessments goal tree. Specialists are invited to evaluate the structure of the proposed model as a whole and make a proposal to include unaccounted relationships in it. It uses a survey. The results of each survey are brought back to the attention of all experts, which allows them to further correct their judgments based on the newly received information. The Delphi method seems to be the most reliable means of obtaining data (this is especially true for information about the future!).

The examination by the Delphi method is carried out according to the scheme shown in Fig. 6.

Rice. 6. Behavior of examination according to the Delphi method

Goal Tree

The most important goal of research in the management of socio-economic systems is effective goal setting. Some modern authors consider goal setting as the most important function of management, which includes:

organizing the efforts of the management system to conduct research in the field of identifying the most promising areas of development;

defining and formulating the goals of the organization, which represent its desired end states;

definition of criteria for assessing the achievement of goals, the basis for comparing the desired results with those achieved.

One of the most effective tools for setting the goals of the management system, as experience shows, is the method of structuring. It is more commonly known as the goal tree method.

The main advantage of this method is that it allows you to identify quantitative and qualitative relationships and relationships between goals, to link their different levels with specific means and timing of achievement.

Goal Tree graphically represents a connected graph, with vertices - goals and edges - links between goals. The graphical representation is used primarily to demonstrate the linkage of top-level goals with sub-goals, which act as a means of achieving these goals. Consists of goal tree from the goals of several levels:

1) the general goal (of the project, organization);

2) goals of the 1st level (main goals);

3) goals of the 2nd level, goals of the 3rd level, and so on up to the required level of decomposition (functionally necessary to achieve the goals of higher levels).

The goal tree method was first proposed by W. Chermen in connection with decision-making problems in industry. Today, in the system analysis of socio-economic systems, the tree of goals is “... this is a structured, built according to a hierarchical principle (distributed by levels, ranked) set of goals of an economic system, program, plan, in which the general goal (“top of the tree”); subgoals of the first, second and subsequent levels subordinate to it (“tree branches”). The name "goal tree" is due to the fact that a schematically represented set of goals distributed over levels resembles an inverted tree in appearance.

The goal tree method is focused on obtaining a complete and relatively stable structure of goals, i.e. such a structure that has changed little over a period of time with the inevitable changes that occur in any developing system. To achieve this, when constructing variants of the structure, one should take into account the patterns of goal formation and use the principles and methods of forming hierarchical structures of goals and functions. In table. 14 shows four types of interdependencies between goals.

Table 14

Interdependencies Between Goals

When formulating goals, managers should check the built pyramid, constantly asking the question: are there (potentially, in fact) the means, resources to achieve the goals.

In the process of building a tree of goals, it is necessary to carry out the following operations to control the formation of a tree of goals:

1) analysis and evaluation of interdependencies between goals: sub-goals of each level must be independent of each other and cannot be derived from each other;

2) determining the importance of goals (based on logical reasoning and expert assessments);

3) establishment of numerical values of indicators of goals (in accordance with calculations, expert assessments);

4) analysis and evaluation of available resources, their distribution, necessary to achieve each goal;

5) control of the hierarchical structure of goals, which proceeds from the principles:

a) the implementation of the subgoals of each subsequent level is a necessary and sufficient condition for achieving the goal of the previous level;

b) the achievement of the goals of a higher level is impossible without the full achievement of all the goals of the lower levels;

c) the completeness of the reduction, i.e. the number of subgoals of each goal should be sufficient to achieve it;

6) when formulating the goals of different levels, the desired results should be described, and not the ways to obtain them.

Based on the results of the current control of the formation of the tree of goals, all branches of the tree that do not fit into resource restrictions and have low calculated and / or expert estimates should be cut off. Building a goal tree should formally reflect the process of distributing goals across management levels. An example of a goal tree is shown in fig. 7.

Rice. 7. An example of a goal tree

From fig. 7 shows that in order to achieve the general goal "Development of the organization on an innovative basis" it is necessary to implement at least three sub-goals:

– “Transition to innovative technologies”;

– “Improving the organization of production”;

– “Improvement of the enterprise management system”.

To achieve these sub-goals, it is necessary to investigate and analyze the factors influencing their achievement. These are two groups of factors - factors contributing to the achievement of goals (available resources), and factors hindering their achievement (lack of necessary resources). Based on these factors, functional goals are formed (given in tables 15 and 16).

Table 15

Decomposition of goals by factors contributing to the achievement of goals


Second level goals

Objectives of the third level - functional (use of available resources)
Financial investment in new technologies	Updating of technological production regulations	Improving the management structure
Purchase of new technologies for the main production	Introduction of new forms of labor organization	Reducing the number of administrative staff
Improving and adapting technologies that cannot be replaced	Improving wages	Improving the personnel management system
	Implementation of new standards for production activities	Improving the technology for preparing management decisions
	Improvement and optimization of production personnel	Reducing workflow

Table 16

Decomposition of goals by factors hindering the achievement of goals

The general goal is the development of the organization on an innovative basis
Second level goals
Transition to innovative technologies	Improving the organization of production	Improving the management system
The goals of the third level are functional (replenishment of missing resources)
Taking measures to find the missing funds	Develop relationships with subcontractors to eliminate downtime, or change subcontractors	Increasing responsibility for making managerial decisions
Finding ways to acquire the technologies necessary for updating, in the face of artificially created difficulties by competitors	Improvement of the system of norms and prices with a focus on the market	Improving management decision-making procedures
Elimination of inconsistency in design and technological developments	Increasing the culture of production	Timely review of job descriptions
Acquisition of technologies that reduce energy costs

Based on a hierarchy of goals goal tree appropriate plans are developed to achieve the general goal - the development of the organization on an innovative basis.

Questions and tasks for self-control

1) What is a model in management systems research?

2) List the main functions that models perform in the study of control systems in management.

3) How are models classified by the time factor?

4) What are the main groups of requirements for models?

5) What is meant by modeling in management?

6) What are the main reasons for using models in management?

7) What methodological tasks are usually solved by models in management? Describe them.

8) What are the goals of descriptive modeling in management?

9) What is the sequence of building a descriptive model?

10) What is a predictive model, what is it intended for?

11) Describe normative modeling. Why is it used in management? Give examples.

12) What are scenarios in scenario analysis?

13) Why is scenario analysis becoming more and more in demand by economic actors?

14) How is a scenario different from a forecast and a vision?

15) Scenarios are distinguished by the type of tasks. What are the main types of tasks that scenarios solve?

16) Describe the main stages in the development of future scenarios for management purposes.

17) Indicate where in management, according to Mats Lindgren and Hans Bandhold, the scenario development technique can be applied?

18) What is the essence and purpose of the Delphi method? Why is it increasingly used in management?

19) What does goal-setting in management include?

20) Describe the goal tree method.

21) What control operations should be carried out in the process of building a tree?

Practical tasks

1) Think and justify which model would be useful for your research? Develop an outline of this model and present it to the group for discussion.

2) Is it possible to use the scenario method in your study? If possible, develop the necessary scenarios in general terms using the “futures scenario” development algorithm.

3) In the working group, analyze the specific situation proposed by the teacher using the Delphi method.

The Delphi method - this name has a multi-stage method, which provides for the principle of experts giving their personal opinions and adjusting them multiple times after reading the opinions of other professionals and people who understand the matter, until a spread is obtained that satisfies the initially set parameters. Estimates obtained by this method can be both static and single form. Therefore, with a qualitative change in the situation, it is necessary to re-apply to experts.

Reliability of the Delphi Method

It is believed that the reliability provided by the Delphi method is high for forecasting both for several years and for a long period of time. To improve accuracy, the number of experts can vary from 10 to 150. Such a qualitative approach helps to determine the specifics of each individual situation. But reliability can be affected by the fact that a subjective factor is of significant importance, which can distance the forecast from reality.

Scope of the Delphi method

Delphi is a method for forecasting and forward planning, used in cases where there is no reliable data regarding the problem under study, a question has arisen that can be given several answers, and none of them is one in which everyone is fully confident. It can also be used in industry, if the task is very much dependent on fundamental sciences, in order to avoid possible problems.

Prerequisites for use and difficulties of application

The following factors may serve as preconditions:

Novelty, and with it the complexity of the problem.
The probabilistic nature of the information provided about the problem.
Insufficiency or inaccuracy of information that relates to the problem.

The difficulties of its application and the resulting information are greater than the prerequisites. There are 5 of them:

Inability to get an accurate forecast.
It is unrealistic to conduct an expert verification of the data obtained due to the fact that the method itself provides for the participation of experts.
The presence of factors beyond the control of the party making the final decision.
The variability of choosing one among several ways of approaching a problem and the need to choose only one.
The incompleteness of information, as well as the inability to process it and translate it from a qualitative level into a quantitative one.

Organization of the examination itself

The organization requires a responsible and balanced approach, because the duration of the survey, as well as the result obtained, quite strongly depends on it. The survey can be organized both by a company that is interested in its result, and by a specialized firm involved in organizing them.

The examination is organized in 6 stages:

Definition of tasks and goals that will be faced by experts.
The choice of the type of procedure for conducting the examination itself.
Selection of experts who will take part in it.
Organization of the examination itself.
Receiving and processing information.
Making a final decision based on the results of the examination.

Determination of the tasks and goals pursued in the examination

The Delphi method implies that initially a problem must be posed on which experts can give their opinion. To do this, it is necessary to provide data on its background, consider the need to solve or predict it, and conduct initial conversations with all representatives of the persons interested in it. The main thing that happens during the definition of the goal is the elimination of secondary or even imaginary problems. Because of this feature, it is not necessary to keep the problem hidden, it must be made public, even if only in a limited circle.

When the problem is identified, it is necessary to know its boundaries, as well as external and internal factors that can affect it. Especially for this, the previously selected question is detailed by creating sub-questions. In this case, you should limit the topics of sub-questions only so that they help answer the main question. In some cases, sub-questions are asked only if it is impossible to obtain an answer to the main one without them. In this case, it must be taken into account that detailing is a double-edged sword. On the one hand, it helps to get a more specific and detailed answer. But on the other hand, the similarity of expert opinions decreases with each step of detailing, and it becomes more and more difficult to bring them all to some general answers.

What are the Delphi classes?

This method has two classes of problems posed:

The first class includes those that have sufficient information potential to solve them. A clear advantage of this class is that within its framework a very specific answer to the problems posed can be given and the accuracy of the forecast is very high.
The second class includes those problems in the solution of which experts cannot be provided with enough information to solve them. The result of this class of expertise must be handled with great care.

Individual survey

The organizers also choose the methods of conducting the survey itself with expert persons. The peculiarity of the individual is that the interviews are taken separately from each expert. This approach allows you to use to the maximum all the knowledge of an expert and get acquainted individually with his point of view.

Group survey

This type of survey uses the exchange and familiarization with the opinions of other experts. During a group survey, everything said is corrected, points that were missed by individual experts are taken into account. With this possibility of questioning, there are also disadvantages: for people, even educated people, it is publicly difficult to abandon the point of view put forward by the group earlier because of psychological weakness. There may also be psychological or individual intolerance of individuals, which does not bode well. This approach in the Delphi method is rarely used to avoid its negative properties.

Characteristics of the Delphi method

Features that characterize the conduct of a survey in which the Delphi method is used:

Anonymity of recognizable opinions among experts.
Controlled processing of the received data, communication is carried out by a group of analysts, but after each round of the survey, the results are provided to other experts.
A group response that summarizes the individual opinion of each expert.

During the survey, the expert environment is divided into two groups. Some of them are mainly engaged in generating ideas, while others analyze and criticize them. At the same time, any criticism is welcomed during the discussion, as well as any idea put forward. The organizers are interested in the discussion being as large as possible and the result turned out to be more correct. The discussions are led by a moderator who should not feel sympathy for certain individuals, and should not be interested in any other results, except for obtaining the most correct information. The leader should not allow the survey to flow into a conversation, a competition for wit, in case of deviating from the topic, return the survey to the right track. All this should happen without imposing the personal opinion of the leader himself, as well as without focusing experts on some

Popular in category: