Minimising the expectation value of the procurement cost in electricity markets based on the prediction error of energy consumption

In this paper, we formulate a method for minimising the expectation value of the procurement cost of electricity in two popular spot markets: {\it day-ahead} and {\it intra-day}, under the assumption that expectation value of unit prices and the distributions of prediction errors for the electricity demand traded in two markets are known. The expectation value of the total electricity cost is minimised over two parameters that change the amounts of electricity. Two parameters depend only on the expected unit prices of electricity and the distributions of prediction errors for the electricity demand traded in two markets. That is, even if we do not know the predictions for the electricity demand, we can determine the values of two parameters that minimise the expectation value of the procurement cost of electricity in two popular spot markets. We demonstrate numerically that the estimate of two parameters often results in a small variance of the total electricity cost, and illustrate the usefulness of the proposed procurement method through the analysis of actual data.


Introduction
In recent years, many power exchanges, such as the Japan Electric Power eXchange (JEPX), Amsterdam Power Exchange, European Power EXchange Spot and PJM Interconnection L.L.C., have been involved in the transfer of energy. These power exchanges offer platforms for trading several electricity markets. We focus our attention to two popular spot markets: day-ahead and intra-day. The day-ahead market trades electricity one day before delivery, whereas the intra-day market trades electricity on the day of delivery.
In practice, it is important for bidders to make a profits as frequently as possible. An accurate forecast of the unit price of electricity may be helpful to increase profits. Many researchers use statistical models or machine learning to forecast the unit price of electricity (e.g., [3], [7], [10], [1] and [14].) Another method to increase profits is strategic bidding, that is, a simulation of the decision-making process of participants in a market. For example, [8] studied a methodology to obtain the optimal bidding strategy for generation companies that participate in electric power markets. Additionally, [13] considered a coevolutionary approach to estimate individual and cooperative strategies of buyers in a power market.
The total electricity cost also depends on the prediction accuracy of energy consumption because the amount of electricity traded in markets is determined by the result of a prediction. As described above, electricity is mainly traded in the day-ahead and intra-day markets. Trading in these two markets should occur in a particular order: bidders trade the estimated total amount of electricity in the day-ahead market and then trade in the intra-day market if the estimated amount of electricity traded in the day-ahead market is not expected to supply the total demand. The unit price of the day-ahead market is usually less than that of the intra-day market. Thus, high prediction accuracy in the day-ahead market may increase profits.
In practice, however, the prediction in the day-ahead market can often be unstable because of the uncertainty of energy consumption. As a result, the procurement cost of electricity highly depends on the prediction error of energy consumption. Therefore, the amount of electricity traded in the two markets must be determined according to the distribution of the prediction error. A few researchers have discussed bidding strategies based on the prediction error from a wind power producer's point of view [2] and [9]. These authors minimised the expectation value of the total cost of electricity demand.
However, the methods proposed by [2] and [9] have three critical problems. First, the authors estimated the expectation value of the total electricity cost not according to the distribution of the prediction error but according to summary statistics of historical aggregate energy output data, which implies that their methods can lead to an inappropriate expectation value of the total electricity cost. The second problem is that the variance of the total electricity cost, which is calculated by a distribution of the prediction error, cannot be obtained using the authors' methods. Third, the authors' methods cannot be applied to multiple electricity markets that are traded in a particular order because their methodologies were developed based on a single market.
To overcome the aforementioned problems, we formulate a method for minimising the expectation value of the procurement cost of electricity. Our proposed method can be used when the unit prices (or the expectation values of the unit prices) of electricity purchased in the two markets and those of supplemental electricity purchased at the time of delivery are provided beforehand (for research on the prediction of the spot price, see the references in [3] and [10].) The distributions of the prediction error of the electricity demand traded in both day-ahead and intra-day markets are taken into account. A numerical integration or Monte Carlo simulation can be used to obtain the expectation value of the total electricity cost. For the Monte Carlo simulation, the variance of the cost can also be computed. The expectation value of the total electric-ity cost (i.e., the procurement cost) is minimised over two parameters, which modify the amounts of electricity to be purchased in the two markets. To show the effectiveness of our proposed procedure, we compute the expectation value and variance of the total electricity cost for various values of tuning parameters when the prediction error follows a normal distribution. We demonstrate numerically that our proposed method results in a smaller variance of total electricity cost than a procurement method that does not take the prediction errors into account. The usefulness of the proposed procedure is illustrated through the analysis of actual data.
The remainder of this paper is organised as follows: in Section 2, we explain the day-ahead and intra-day markets and the penalty paid for the purchase of supplemental electricity at the time of delivery. In Section 3, we present our procurement method that prescribes the amounts of electricity purchased from the two markets, derive the expected total cost of electricity and express the expectation value of this cost in terms of probability distribution functions of the prediction error. In Section 4, we report the results of a numerical calculation in which we minimise this expectation value over the two parameters, assuming that the distributions of prediction errors follow normal distributions with mean zero. In Section 5, we apply our method to actual data and demonstrate that the procurement cost can be improved by employing our procurement method.
2 Electricity markets and the penalty paid for supplemental purchase In this section, we explain the day-ahead market, intra-day market and the penalty for supplemental purchase. JEPX is one of the most popular electric power exchanges in Japan. It operates several electricity exchange markets. We consider participation in two of these markets: day-ahead and intra-day. In the day-ahead market, electricity to be delivered on the following day is purchased, and in the intra-day market, electricity to be delivered on that day is purchased. The delivery schedule of electricity for a given day is divided into 48 periods that consist of 30 minutes each. The first period is from midnight to 0:30 a.m. and the final period is from 11:30 p.m. to midnight. For example, if some amount of electricity is procured for the 25th period in the day-ahead market, then that amount is designated for delivery on the next day between noon to 0:30 p.m, whereas if it is procured for the 25th period in the intra-day market, then it is designated for delivery on that day between noon and 0:30 p.m.
Suppose we are purchasing electricity for delivery on a given day. We refer to this day as the "day of delivery." Then, if for delivery in the t-th period, amount e 1 is procured in the day-ahead market on the previous day and amount e 2 is procured in the intra-day market on the delivery day, then we have e 1 + e 2 of electricity available for delivery in the t-th period on the delivery day. The actual amount of demand for electricity during this period may be more or less than this amount. We denote this demand by f (t).
Next, we discuss the procedures followed to manage a mismatch between the amount procured and amount demanded in the cases of f (t) > e 1 + e 2 and f (t) < e 1 + e 2 . When f (t) > e 1 + e 2 , the surplus amount of electricity, e 1 + e 2 − f (t), is acquired by an electric company (a general electricity transmission and distribution company). In the case of f (t) < e 1 + e 2 , this company provides the (necessary) supplemental amount f (t) − e 1 + e 2 and charges a penalty for this service.
In this paper, considering electricity procured for a given period of delivery, we assume the following relations among unit prices: day-ahead unit price < intra-day unit price < penalty unit price . (1) However, there are cases in which the penalty unit price is less than one or both market unit prices, and the above inequality does not hold: The intra-day unit price is less than the day-ahead unit price. Assuming that the penalty unit price is the highest price is equivalent to assuming that the market is functioning properly in accordance with the "planned-value power balancing system" (see e.g., [6]). Indeed, if the penalty unit price was lower than the market unit prices, then foregoing participation in the markets and simply procuring all electricity as supplemental purchases and paying the resulting penalties would result in the lowest procurement cost. However, this would place a large burden on power generation facilities; that is, it would not be in accordance with the planned-value power balancing system. Additionally, relying mainly on the intra-day market for procurement would place a similarly large burden on power generation facilities. For these reasons, we assume the relations (1). However, it should be noted that this assumption does not have a significant effect on the results obtained in this work. We only use this assumption as a condition in the simulations reported below. With this assumption, following the method proposed, most electricity is procured in the day-ahead market. Assuming that we do not have access to a large storage cell, we are not able to store electricity, and for this reason, we wish to avoid the scenario in which we procure an amount of electricity that exceeds demand. However, given the aforementioned relation among unit prices, we also wish to avoid paying penalties. It is necessary that we attempt to optimally balance these two undesirable scenarios because there is always uncertainty in our prediction of demand. For this reason, devising a procurement method for minimising the procurement cost that accounts for this uncertainty is very important for the proper functioning of electricity markets.

Proposed method
In this section, we explain the proposed procurement method and derive the procurement cost for the total quantity of electricity procured using this method for delivery during the t-th period of the delivery day. Then, we express the expectation value of this cost in terms of probability distributions that represent the errors in the demand predictions.

Procurement method
The proposed procurement method takes as its input predictions for demand, f (t), in the t-th period on the day of delivery. These predictions are made at two times: once during the day-ahead market and once during the intra-day market. With these predictions, using two parameters A and B that account for the effect of the error in these predictions and the differences among the unit prices, our method yields optimal procurement amounts in the day-ahead and intra-day markets.
Let g(t) denote the prediction for f (t) made at the time of procurement in the day-ahead market; hereafter, the "previous-day prediction." Then, to hedge against the uncertainty in this prediction, we add some amount A(t) and procure the amount g(t) + A(t) in the day-ahead market. The quantity A(t) is to be chosen in such a manner that optimally accounts for the uncertainty in g(t) and the differences among the unit prices for the three approaches of procuring electricity.
Next, let h(t) denote the prediction for f (t) made at the time of procurement in the intra-day market; hereafter, the "same-day prediction." Then, in the intra-day market, electricity is procured such that the sum of the amounts procured in the two markets is h(t) + B(t), where B(t) is a quantity that plays the same role here as A(t) plays in the day-ahead market. However, in the case that h(t) + B(t) is less than the amount of electricity procured in the day-ahead market, this is obviously not possible. In this case, no electricity is procured in the intra-day market.

Procurement cost
In this subsection, we derive the total cost of the amount of electricity procured using the method described in Section 3.
Let a denote the unit price for the t-th period in the day-ahead market. Because we always consider the t-th period in the following discussion, from this point, we will generally omit the argument t for the various quantities considered. Then, because the amount procured in the day-ahead market is g + A, the cost of this procurement is Next, let b denote the unit price for the t-th period in the intra-day market. Then, because the sum of the amounts procured in the two markets is h + B, if g + A ≤ h + B, then the amount procured in the intra-day market is h + B − (g + A), whereas if g + A ≥ h + B, then the amount procured in the intra-day market is zero. Thus, the cost of procuring electricity in this market is where δ( * ) = 1 if condition * is satisfied; otherwise, δ( * ) = 0 . C 1 + C 2 is the combined cost of electricity procured in the two markets. To obtain the total procurement cost, we must also calculate the penalty paid for supplemental electricity, which we denote by C 3 . There are two separate scenarios in which we must purchase supplemental electricity: that in which the condition g + A ≤ h + B ≤ f is satisfied and that in which the condition h + B < g + A ≤ f is satisfied. Then, with c denoting the unit price for the penalty, the total amount we pay for the penalty is (4) Collecting the above results, the total procurement cost for electricity delivered in the t-th period on the day of delivery is C = C 1 + C 2 + C 3 .

Expectation value of the procurement cost
In this subsection, we obtain an expression for expectation value E[C] of procurement cost C calculated in Subsection 3.2 in terms of probability distributions that represent the errors in the demand predictions. G = f −g and H = f −h are assumed to be random variables, and we denote the distribution functions of these variables by P G (x) and P H (y), respectively. We assume that f and (a, b, c) are independent. If the demand f is much smaller than the total trading volume of the electricity market, then this assumption would be valid.
The expectation value of procuring electricity in the day-ahead market is Next, we obtain the expectation value of procuring electricity in the intraday market, C 2 . For this purpose, we first rewrite C 2 in terms of G and H: From this, we obtain Finally, we obtain the expectation value of the penalty, C 3 , similarly to that of C 2 . First, we rewrite C 3 in terms of G and H: We then immediately obtain We have thus obtained the following expression for the expectation value of total cost C for electricity to be delivered in the t-th period on the day of delivery: Then, we know that the values of A and B that minimise , P G (x) and P H (y); that is, even if we do not know the predictions g and h, we can determine the values of A and B that minimise E[C] if we know the prediction error distributions P G (x) and P H (y).

E[C] in the case that G and H are normally distributed
In this subsection, we seek the expectation value of the procurement cost E[C] in the case that the errors in the demand predictions, G and H, are normally distributed random variables with mean zero.
We assume that the difference between the previous-day demand prediction and actual demand G is normally distributed, with mean zero and variance σ 1 (t) 2 = σ 2 1 , and that the difference between the same-day demand prediction and actual demand H is normally distributed, with mean zero and variance and the expectation value of C 3 is expressed as We thus have the following result for the expectation value of the procurement cost: Thus, we observe that by carefully choosing the procurement amounts, we can both minimise the expectation value of the procurement cost and increase its stability.

Numerical calculation of E[C]
With the procurement conditions fixed as we conducted numerical computations in which we determined the expectation value of C = C 1 + C 2 + C 3 . First, we explain why we chose the above conditions. We chose the value √ 3 for the standard deviation σ 1 because, among electric companies, the target level of precision for the previous-day demand prediction is within ±3% [12, p. 53]. Then, because the precision of the same-day prediction is generally higher, we chose the value σ 2 = √ 2. Next, for the expectation value of the unit price of the penalty, we chose E[c] = 3 because this unit price is often three times or more greater than the day-ahead unit price [11, p. 425]. From the above results, we observe that with the procurement conditions given above, we can reduce E[C] by procuring slightly more than the amount  of the previous-day demand prediction and slightly less than the amount of the same-day demand prediction.

Stability of E[C]
Generally, even if we have a method to minimise the expectation value of the procurement cost, this method is not practically useful when the convergence to the expectation value is slow due to the variance of the cost is too large. This is because it entails a large financial risk for companies to use this procurement method when procurement costs continue to be greater than their expected values even if it is a short term. In the previous subsection, we reported the values of A and B for which the minimum value of E[C] is achieved. In this subsection, we investigate the unbiased variance of C, V [C], achieved using these values of A and B. Through this investigation, we determine that not only do these values minimise E[C], but they also result in a relatively small value of V [C]. First, we conducted 10 6 iterations of a Monte Carlo simulation of the procurement process for each value of (A, B) on a grid of mesh size 0.1 × 0.1. From these simulations, we obtained the variance that corresponded to each such value of (A, B). We note that these simulations use equation (2), (6) and (8). Here, we do not use the equation (13).
The conditions of the simulations were set as follows: For the previous-day and same-day demand predictions, we used normal distributions with mean 0: The We study the dependence of C on A and B, and for this reason, we express this dependence explicitly by writing C as C(A, B).
In the range of values of (A, B) considered in our simulations, the variance was minimised at   From the above results, we determine that with the proposed method, and under the procurement conditions stated above, we were able to purchase electricity with a more stable price than in the case that procurement was made simply in accordance with demand predictions. This effect can be understood as resulting from a suppression of the penalty paid because of an increase in the amount of electricity procured in the day-ahead market. Additionally, as can be observed in Fig. 2, when A was large, the value of B had almost no effect on the variance of C. This can be understood as resulting from the fact that because there is little difference between the variances of the previous-day and same-day predictions, it is usually possible to procure the necessary electricity in the day-ahead market.

Simulations using actual data
In this section, we report the results of simulations of the proposed procurement method using actual data. From these simulations, we determine that this method minimises the procurement cost. We use kilowatt-hours (kWh) as the unit of electricity and Japanese yen as the unit of cost.
The data presented in Table 2 are the actual values of demand f (t, d) experienced by a particular facility in Kasuga City, Fukuoka Prefecture, on 19 weekdays corresponding to the values d = 1 to 19, in January 2017 (days 1-3 correspond to January 4 to January 6, days 4-7 correspond to January 10 to January 13, days 8-12 correspond to January 16 to January 20, days 13-17 correspond to January 23 to January 27 and days 18-19 correspond to January 30 to January 31) for the periods t = 20 to t = 26 (from 9:30 a.m. to 1:00 p.m.) We limit the time zone from t = 20 to t = 26 because the prediction errors of this time zones are large. In actual electricity markets, in the day-ahead market, the unit of traded electricity is 500 kWh [4, p. 7], whereas in the intra-day market, it is 50 kWh [4, p. 12]. However, these units of traded electricity are larger than the demand of the facility. Therefore, in this paper, we assume the scenario in which the amount of electricity traded in the markets can be freely adjusted and smaller amounts can be traded.
Previous-day predictions g(t, d) and same-day predictions (made 90 minutes before each time period in question) h(t, d) for the actual demands are presented in Tables 3 and 4. In Tables 5, 6 and 7, we present day-ahead market unit prices a(t, d), intraday market unit prices b(t, d) and penalty unit prices c(t, d) for the same days and time periods as in the previous tables. The prices listed are the area prices for the Kyushu region. Additionally, for each intra-day unit price b(t, d), we use the average price during the intra-day market for this time period. These data were obtained from the JEPX website on January 31, 2018. To derive the optimal values of A and B, we need predictions of the unit prices and error distributions for the demand predictions. However, we do not presently have a systematic method for generating predictions of unit prices. For example, there is research on price prediction [1]; however, the trading method of JEPX changes frequently, which causes fluctuations in prices, so price prediction does not apply to recent data.
Instead, we use the following simple prescription for the predictions of the day-ahead unit priceâ(t, d), intra-day unit priceb(t, d) and penalty unit pricê c(t, d). For the time periods 20-24, we usê and for the time periods 25 and 26, we usê where x represents a, b and c. We chose these prescriptions for the predictions because, as can be observed from the Tables 5-7 listing actual prices, the prices during the time periods 20-24 vary significantly from day to day, whereas the prices in periods 25 and 26 do not. The predictions obtained in this manner for each time period are presented in Tables 8, 9 and 10.
For the distribution functions of the errors in the demand predictions, we use normal distributions with mean zero. Typically, the demand prediction is performed using normal linear regression modelling. In particular, leastsquares estimation (LSE) produces a distribution of the prediction error whose mean value is zero because the LSE yields an unbiased estimate of the mean of prediction value.
In practice, the values of A and B are determined before the trade in both day-ahead and intra-day markets. Because both A and B depend on the distributions of the prediction error, these distributions must be estimated before the trade in the two markets. However, the prediction in the intra-day market is performed after the trade in the day-ahead markets; hence, the distribution of the prediction error in the intra-day market cannot be obtained before the trade in the day-ahead market. Therefore, we need to estimate the distribution of the prediction error without the prediction in the intra-day market to determine the values of A and B.
The estimates of the variances of the prediction error are given as follows: For time periods 20-24, to estimate the error variance in the previous-day prediction V 1 (t, d), we use the variance of the predicted values of linear regression [5]. As an estimator of the variance of the prediction error in the same-day prediction, we useV We present the estimates of the variances obtained in Tables 11 and 12. Using the above values for the predictions of unit pricesâ(t, d),b(t, d) and c(t, d), and the variances of the errors in the demand predictionsV 1 (t, d) and V 2 (t), we numerically determined the values of A and B that minimised E[C]. However, in the caseb ≤â, we chose A = 0, and in the caseĉ ≤b, we chose B = 0 to maintain the planned-value power balancing system because procuring large amounts of electricity in the intra-day market or at the time of delivery places a great burden on power generation facilities, and hence is not a maintainable market scenario.
Employing the optimised values of A and B that appear in Tables 13 and 14, and performing the procurement simulations, we obtained a total procurement cost of 51, 949.95 yen. We compared this cost with those obtained in two other scenarios. First, if the previous-day prediction of the demand was perfect and we procured all the electricity in the day-ahead market, then we would obtain a total procurement cost of 51, 140.72 yen. Second, using the actual predictions g and h, if we simply procured electricity in accordance with these predictions (i.e., always using (A, B) = (0, 0)), we would have a procurement cost of 52, 225.97 yen. Thus, the procurement cost achieved with our method was 809.23 yen higher than in the ideal case of perfect demand prediction and 276.02 yen lower than in the case that the ordinary procurement method was followed. This result show that the procurement cost can be reduced by about 0.5%. It is important to reduce the procurement cost by 0.5% for electric power companies.
With the assumption that individual procurements can be made in any amount, the simulations conducted using actual data demonstrated that our method minimised the procurement cost. Because the simulation time was short and the simulation scale was small, the cost reduction was small. But, if the simulation time was long and the simulation scale was large, then a significant cost reduction could be expected.

Conclusion
We formulated a method for minimising the expected procurement cost of electricity and we reported the results of simulations of the proposed procurement method using actual data. From these simulations, we found that this method minimised the procurement cost. Based on the above discussion, to reduce the procurement cost it was necessary to estimate not only the accuracy of predictions but also the prediction error distributions. In this paper, the prediction error distributions were assumed to be normal distributions, but the actual prediction errors did not always become normal distributions. To achieve further cost reduction, an estimation of the prediction error distributions is indispensable.

Appendix: Simulations investigating the effect of altering the procurement conditions
In this appendix, we consider a variation of the conditions (14)    Using the parameter values A = 0.6 and B = −2, which were the optimal parameter values found using the procurement conditions given in Section 4.2, as standards for comparison, we found that as the precision of the previous-day demand prediction decreased, it was advantageous to increase both A and B.
Next, we consider the case in which the procurement conditions are as in the conditions (14), except that σ 2 is changed from   Thus, we observe that when the precision of the same-day demand prediction improved, it was advantageous to decrease A and increase B from their standard values.
As the next scenario, we consider the case in which the procurement conditions are as in the conditions (14), except that the intra-day unit price b is decreased from 2 to 1. Thus, we observe that when the procurement cost in the intra-day market decreased and approached that in the day-ahead market, it was advantageous to decrease A and increase B from their standard values.  As the fourth scenario for comparison, we now consider the case in which the procurement conditions are as in the conditions (14), except that intra-day unit price b is increased from 2 to 2.8. The results of these simulations are displayed in Figs   Thus, we observe that when the intra-day unit price increased and approached the penalty cost, it was advantageous to increase A and decrease B from their standard values.
As the fifth scenario, we consider the case in which the procurement conditions are as in the conditions (14), except that the day-ahead unit price a decreases from 1 to 0.5. The results of these simulations are displayed in Figs   Thus, we observe that when the day-ahead unit price decreased, it was advantageous to increase A and decrease B from their standard values.
As the sixth scenario, we consider the case in which the procurement conditions are as in the conditions (14), except that penalty unit price c increases from 3 to 3. Thus, we observe that when the penalty unit price increased, it was advantageous to increase A and decrease B from their standard values.  In Table 1, we summarise the changes undergone by the optimal values of A and B relative to their standard values because of the various types of changes to the procurement conditions considered above.
A B precision of previous-day demand prediction decreases increase increase precision of same-day demand prediction increases decrease increase intra-day unit price approaches day-ahead unit price decrease increase intra-day unit price approaches penalty unit price increase decrease day-ahead unit price decreases increase decrease penalty unit price increases increase increase       Table 12: Estimated values of the variance of the error distributions for the same-day demand predictions.