Nowadays, the functioning of any sector and the development and survival of todayʼs civilization is almost unthinkable without the reliable power system, which is just over a century old. Initially, isolated (island) power systems emerged that began to interconnect with the expansion, so today, for example, there is a common European power network. This network is certainly not perfect, as there are technical limitations in transmission capacity within and between countries. It is important to emphasize that the traditional power system was based on thermal power plants (coal, gas and even oil – fuel oil) and run-of-the-river and storage hydropower plants. Initially, the electricity market was also not established, and electricity is a good that differs in many respects from other tradable goods. After the process of deregulation, liberalization and restructuring of vertically integrated monopolistic power utilities, several decades ago, electricity pools were established, access of smaller, private investors in the system was allowed. In parallel with these events, Europe, as well as other world regions and large countries, have noticed the problem of climate change and global warming, which is mainly due to large amounts of CO2 emissions, a large share of which originates from the power system. Based on these changes, global and more specifically European regulations have been established, and strategic documents have been drafted, that encourage sustainable development, primarily through the use of renewable energy sources, mostly wind and solar power plants. The problem at first was that these, then unconventional, technologies were not sufficiently developed, and therefore not really competitive with traditional power plants. vii Therefore, sustainable renewable energy sources are encouraged in various ways, mostly through the system of incentives (feed-in tariffs - guaranteed purchase price of electricity). Due to the encouragement of technological development and reduction of investment costs (for wind and solar power plants, fixed costs are practically the only costs, because the cost of energy sources is approximately equal to zero) there has been an exponential increase in the penetration of renewable sources into the power sector. Therefore, in one part of this dissertation, the levelized cost of energy (LCOE) of renewable energy power plants (RES) is investigated and compared with the LCOE of traditional power plants. However, in addition to the benefits they bring, these renewable energy sources also bring certain challenges to the power systems. In the dissertation, based on the model of the power system of the Republic of Croatia, which is modeled in the optimization tool PLEXOS, the mentioned problems are identified and a solution for them is proposed. The main problem of wind and solar power plants is the unpredictability of their production, i.e. they are not dispatchable in the same way as traditional power plants, and one of the prerequisites for the normal functioning of the power system is the balance between consumption and electricity production. This in turn requires additional sources of flexibility in the power system. Therefore, the dissertation deals with different ways of providing additional flexibility, such as expansion of transmission and distribution systems (capital intensive), gas thermal power plants (carbon footprint), pumped storage hydropower plants (lack of favorable locations for the construction of new plants) and battery systems. Of all these balancing technologies, battery storage systems have proven to be the most promising. Therefore, the main part of the research deals with the competitiveness of battery systems in the existing power system. The basic criterion of cost-effectiveness of a battery is its LCOE, which is compared with the average market price, and based on this comparison, a decision is made on the cost-effectiveness of the project of investing in the battery system. It is important to emphasize that for the needs of analysis and research the dissertation defines two scenarios of future penetration of wind and solar power plants: optimistic (high RES penetration - highest amounts based on strategic documents of the Republic of Croatia) and pessimistic (twice slower annual increase in RES penetration than in the optimistic scenario). In order to test the set hypotheses and research questions, over 20 scenarios were created, which differ in the size of the battery system integrated into the power system and the speed of future RES penetration in the period from 2021 to 2030 (optimistic and pessimistic scenario). The results show that assuming that the battery used to balance the system, pays the market price for charging, and to receive the market price for discharge, cannot make a profit in the projected lifetime. In other words, certain incentives are needed for new battery systems. The results also indicate that the integration of batteries in the pessimistic scenario reduces the social welfare, i.e. there are no means to stimulate battery systems in this case. In contrast, in the optimistic scenario, the integration of batteries increases the social welfare, i.e. there are means to stimulate battery systems.