format: text

text:
# 1 Introduction

## 1 Introduction

As discussed by Bahadur , Fisher information characterizes the limit of the decreasing rate of the tail probability of the optimal estimator. However, when the Kullback-Leibler (KL) divergence diverges or Fisher information cannot be defined, this cannot be applied. As an alternative information quantity between two probabilities, we can use the relative Rényi entropies $`I^s(pq):=\mathrm{log}p^s(\omega )q^{1s}(\omega )𝑑\omega (0<s<1)`$, which play an important role in simple hypothesis testing. (Hoeffding , Chernoff ) In general, for a probability distribution family $`\{p_\theta |\theta \}`$, the relative Rényi entropies $`I^s(p_\theta p_{\theta +ϵ})`$ tend to $`0`$ when $`ϵ`$ goes to $`0`$, but their order is not necessarily $`ϵ^2`$. In this paper, we treat a large deviation theory, which can be applied to such a case. The importance of the relative Rényi entropies $`I^s(p_\theta p_{\theta +ϵ})`$ was pointed out by Akahira  from the viewpoint of information loss in a non-regular family. Although the limit distribution of the maximum likelihood estimator (MLE) in a non-regular family has been extensively discussed , large deviation theory has not been discussed sufficiently in this respect.

In a large deviation theory of the parameter estimation, we usually focus on the first exponential rate

$`\beta (\stackrel{}{T},\theta ,ϵ):=lim\; inf{\displaystyle \frac{1}{n}}\mathrm{log}p_\theta ^n\{|T_n\theta |>ϵ\}`$ (1)
for a sequence of estimators $`\stackrel{}{T}=\{T_n\}`$, which is simply called an estimator in the following. Of course, when $`\beta (\stackrel{}{T},\theta ,ϵ)`$ is large, the estimator $`\stackrel{}{T}`$ is better. We discuss its maximization at the limit $`ϵ0`$. As is explained in Section 2, Bahadur focused on the bound $`\alpha (\theta )`$ of the first exponential rate at the limit $`ϵ0`$, whose definition is precisely given in Section 2. He derived its upper bound from the viewpoint of Stein’s lemma in simple hypothesis testing.

The main purpose of this paper is to extend the bound $`\alpha (\theta )`$ and derive its upper bound in a more general setting. Indeed, depending on how the limit $`ϵ0`$ is obtained, we can extend the bound $`\alpha (\theta )`$ in two ways. This difference can be regarded as the difference between the point estimation and the limit of the interval estimation. Bahadur’s original theory concerned the point estimation, but it can also be applied to the limit of the interval estimation, as is explained in Section 2. Therefore, from the two ways of extending the bound $`\alpha (\theta )`$, we can define two generalizations $`\alpha _1(\theta )`$ and $`\alpha _2(\theta )`$ of Bahadur’s bound $`\alpha (\theta )`$. In Section 3, we give the respective upper bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$, and check that our result contains Bahadur’s results as a special case in which the two upper bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$ coincide. Proofs of the two inequalities $`\alpha _1(\theta )\overline{\alpha }_1(\theta )`$ and $`\alpha _2(\theta )\overline{\alpha }_2(\theta )`$ are given in Section 4.

In Sections 5 and 6, we also discuss the attainability of upper bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$, and calculate them for location shift families. In Section 5, we derive several formulae for the first exponential rates of the maximum likelihood estimator (MLE) and some estimators consisting of order statistics under certain assumptions. In Section 6, using a formula for relative Rényi entropies given by Hayashi , we calculate the two upper bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$, and derive a sufficient and necessary condition for their coincidence. Using formulae given in Section 5, we check that these bounds are attainable in some special cases. Our examples in Section 6 contain location shift families generated by a Weibull distribution, gamma distribution, beta distribution, and uniform distribution.

## 2 Bahadur theory

In this section, we begin by summarizing the results reported by Bahadur , who discussed the decreasing rate of the tail probability in the estimation for a distribution family. Given $`n`$-i.i.d. data $`\omega _1,\mathrm{},\omega _n`$, the first exponential rate $`\beta (\stackrel{}{T},\theta ,ϵ)`$ of the estimator $`\stackrel{}{T}=\{T_n\}`$ is written as

$`\beta (\stackrel{}{T},\theta ,ϵ)=\mathrm{min}\{\beta ^+(\stackrel{}{T},\theta ,ϵ),\beta ^{}(\stackrel{}{T},\theta ,ϵ)\},`$
where the exponential rates of half-side error probabilities are given by

$`\beta ^+(\stackrel{}{T},\theta ,ϵ)`$ $`:=lim\; inf{\displaystyle \frac{1}{n}}\mathrm{log}p_\theta ^n\{T_n>\theta +ϵ\}`$
$`\beta ^{}(\stackrel{}{T},\theta ,ϵ)`$ $`:=lim\; inf{\displaystyle \frac{1}{n}}\mathrm{log}p_\theta ^n\{T_n<\theta ϵ\}.`$
When an estimator $`\stackrel{}{T}=\{T_n\}`$ satisfies the weak consistency

$`p_\theta ^n\{|T_n\theta |>ϵ\}0ϵ>0,\theta \mathrm{\Theta },`$
using the monotonicity of KL-divergence, we can prove the inequality

$`\beta (\stackrel{}{T},\theta ,ϵ)\mathrm{min}\{D(p_{\theta +ϵ}p_\theta ),D(p_{\theta ϵ}p_\theta )\}.`$ (2)
Note that if, and only if, the family is exponential, there exists an estimator attaining the equality (2) at $`\theta \mathrm{\Theta },ϵ>0`$. Therefore, for a general family, it is difficult to optimize the first exponential rate $`\beta (\stackrel{}{T},\theta ,ϵ)`$.

We usually introduce the second exponential rate as another optimized value:

$`\alpha (\stackrel{}{T},\theta ):=\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^2}}\beta (\stackrel{}{T},\theta ,ϵ).`$ (3)
In this case, when the Fisher information $`J_\theta `$ satisfies the condition

$`J_\theta :={\displaystyle l_\theta (\omega )^2p_\theta (d\omega )}=\underset{ϵ0}{lim}{\displaystyle \frac{2}{ϵ^2}}D(p_{\theta +ϵ}p_\theta ),l_\theta (\omega ):={\displaystyle \frac{d}{d\theta }}\mathrm{log}p_\theta (\omega ),`$
the inequality

$`\alpha (\stackrel{}{T},\theta ){\displaystyle \frac{1}{2}}J_\theta `$ (4)
holds. Moreover, as was proven by Fu , if the family satisfies the concavity of the logarithmic derivative $`l_\theta (\omega )`$ for $`\theta `$ and some other conditions, the MLE $`\stackrel{}{\theta _{ML}}`$ attains the equality of (4). These facts are summarized in the two inequalities

$`\alpha (\theta ):=\underset{\stackrel{}{T}:\mathrm{WC}}{sup}\alpha (\stackrel{}{T},\theta )`$ $`={\displaystyle \frac{1}{2}}J_\theta `$ (5)
$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^2}}\underset{\stackrel{}{T}:\mathrm{WC}}{sup}\beta (\stackrel{}{T},\theta ,ϵ)`$ $`={\displaystyle \frac{1}{2}}J_\theta .`$ (6)
Inequality (5) can be regarded as the bound of the point estimation, while inequality (6) can be regarded as the limit of the bound of the interval estimation because $`sup_{T:WC}\beta (\stackrel{}{T},\theta ,ϵ)`$ corresponds to the bound of the interval estimation with width $`2ϵ`$ of the confidence interval.

## 3 Main results

In this paper, the relative Rényi entropies $`I^s(p_\theta p_{\theta +ϵ})`$ substitute for the KL divergence. Note that the order of $`I^s(p_\theta p_{\theta +ϵ})`$ is not necessarily $`ϵ^2`$ at the limit $`ϵ0`$. However, its order is independent of the parameter $`s`$, as is guaranteed by the inequalities

$`2\mathrm{min}\{s,1s\}I^{\frac{1}{2}}(p_\theta p_{\theta +ϵ})I^s(p_\theta p_{\theta +ϵ})2\mathrm{max}\{s,1s\}I^{\frac{1}{2}}(p_\theta p_{\theta +ϵ}),`$
which are proven in Lemma 12 of the Appendix. In several cases, the order of the first exponential rate $`\beta (\stackrel{}{T},\theta ,ϵ)`$ coincides with the order of the relative Rényi entropies $`I^s(p_\theta p_{\theta +ϵ})`$. In the following, we use a strictly monotonically decreasing function $`g(x)`$ such that $`I^s(p_\theta p_{\theta +ϵ})O(g(ϵ))`$ and $`g(0)=0`$.

Following equations (5) and (6), we define two extensions of Bahadur’s bound $`\alpha (\theta )`$ as

$`\alpha _1(\theta )`$ $`:=\underset{ϵ+0}{lim\; sup}{\displaystyle \frac{1}{g(ϵ)}}\underset{T}{sup}\underset{\theta ϵ\theta ^{}\theta +ϵ}{inf}\beta (\stackrel{}{T},\theta ^{},ϵ)`$ (7)
$`\alpha _2(\theta )`$ $`:=\underset{T}{sup}\alpha _2(\stackrel{}{T},\theta ),`$ (8)
where

$`\alpha _2(\stackrel{}{T},\theta ):=\underset{ϵ+0}{lim\; inf}{\displaystyle \frac{1}{g(ϵ)}}\underset{\theta ϵ\theta ^{}\theta +ϵ}{inf}\beta (\stackrel{}{T},\theta ^{},ϵ).`$ (9)
Note that we take infimum $`inf_{\theta ϵ\theta ^{}\theta +ϵ}`$ into account in (9), unlike (3). As was pointed out by Ibragimov and Has’minskii , when KL-divergence is infinite, there exists a consistent super efficient estimator $`\stackrel{}{T}`$ such that $`\beta (\stackrel{}{T},\theta ,ϵ)`$ and $`lim_{ϵ+0}\frac{1}{g(ϵ)}\beta (\stackrel{}{T},\theta ,ϵ)`$ is infinite at one point $`\theta `$. Therefore, we need to take the infimum $`inf_{\theta ϵ\theta ^{}\theta +ϵ}`$ into account. In this situation, we do not need to limit estimators to weakly consistent ones. As is proven in the next section, we can obtain the following theorems.

###### Theorem 1

When the convergence $`lim_{ϵ0}\frac{I^s(p_{\theta ϵ/2}p_{\theta +ϵ/2})}{g(ϵ)}`$ is uniform for $`0<s<1`$, the inequality

$`\alpha _1(\theta )\overline{\alpha }_1(\theta ):=2^\kappa \underset{0<s<1}{sup}I_{g,\theta }^s`$ (10)
holds, where $`\kappa `$ and $`I_{g,\theta }^s`$ are defined by

$`I_{g,\theta }^s`$ $`:=\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(p_{\theta ϵ/2}p_{\theta +ϵ/2})}{g(ϵ)}}1s0`$ (11)
$`x^\kappa `$ $`=\underset{ϵ+0}{lim}{\displaystyle \frac{g(xϵ)}{g(ϵ)}}.`$
Lemma 10 proven in Appendix C, guarantees the existence of such a real number $`\kappa `$.

Note that the function $`sI_g^s`$ is concave and continuous because the function $`sI^s(p_{\theta \frac{1}{2}ϵ}p_{\theta +\frac{1}{2}ϵ})`$ is concave and continuous. Therefore, when $`I_{g,\theta }^s=I_{g,\theta }^{1s}`$, we have

$`\overline{\alpha }_1(\theta )=2^\kappa I_{g,\theta }^{\frac{1}{2}}.`$ (12)
###### Theorem 2

If the convergence $`lim_{ϵ0}\frac{I^s(p_{\theta ϵ/2}p_{\theta +ϵ/2})}{g(ϵ)}`$ is uniform for $`s(0,1)`$ and $`\theta K`$ for any compact set $`K`$, the inequality

$`\alpha _2(\theta )\overline{\alpha }_2(\theta ):=\{\begin{array}{cc}sup_{0<s<1}\frac{I_{g,\theta }^s}{s(1s)}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}& \text{ if }\kappa <1\\ 2I_{g,\theta }^{\frac{1}{2}}& \text{ if }\kappa =1\\ inf_{0<s<1}\frac{I_{g,\theta }^s}{s(1s)}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}& \text{ if }\kappa >1\end{array}`$ (16)
holds.

In our proofs of these theorems, Chernoff’s formula and Hoeffding’s formula in simple hypothesis testing play important roles.

As was proven by Akahira , under some regularity conditions for a distribution family, the equation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(p_\theta p_{\theta +ϵ})}{ϵ^2}}={\displaystyle \frac{J_\theta s(1s)}{2}},`$ (17)
holds. When we choose $`g(x)=x^2`$, we have $`\kappa =2,I_{g,\theta }^s=\frac{1}{2}J_\theta s(1s)`$, and the relations

$`\overline{\alpha }_1(\theta )`$ $`=4\underset{0s1}{\mathrm{max}}I_{g,\theta }^s={\displaystyle \frac{1}{2}}J_\theta `$
$`\overline{\alpha }_2(\theta )`$ $`=\underset{0s1}{\mathrm{min}}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}={\displaystyle \frac{1}{2}}J_\theta `$
hold. In particular, if the distribution family satisfies the concavity of the logarithmic derivative $`l_\theta (\omega )`$ for $`\theta `$ and some other conditions, the bound $`\frac{1}{2}J_\theta `$ is attained by the MLE. Thus, the relations $`\alpha _1(\theta )=\alpha _2(\theta )=\overline{\alpha }_1(\theta )=\overline{\alpha }_2(\theta )=\frac{1}{2}J_\theta `$ hold.

As a relation between two bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$, we can prove the following theorem.

###### Theorem 3

The inequality

$`\overline{\alpha }_1(\theta )\overline{\alpha }_2(\theta )`$ (18)
holds, and (18) holds as an equality if and only if the equations

$`\overline{\alpha }_1(\theta )`$ $`=2^\kappa I_{g,\theta }^{\frac{1}{2}}`$ (19)
$`2^\kappa I_{g,\theta }^{\frac{1}{2}}`$ $`=\overline{\alpha }_2(\theta )`$ (20)
hold. When $`\kappa 1`$, (18) holds as an equality if and only if equation (19) holds.

When $`I_{g,\theta }^s`$ is differentiable, condition (19) is equivalent to $`\frac{d}{ds}I_{g,\theta }^s|_{s=\frac{1}{2}}=0`$.

## 4 Proofs of main results

In our proofs of Theorems 1 and 2, Chernoff’s formula and Hoeffding’s formula in simple hypothesis testing are essential, and are summarized as follows. Let the probability $`p`$ on $`\mathrm{\Omega }`$ be the null hypothesis and $`q`$ be the alternative hypothesis. When we discuss a hypothesis testing problem concerning $`n`$-i.i.d. data, we call a sequence $`\stackrel{}{A}=\{A_n\}`$ a test, where $`A_n`$ is an acceptance region, which is a subset of $`\mathrm{\Omega }^n`$. The first error probability $`e_1(A_n)`$ and the second error probability $`e_2(A_n)`$ are defined as

$`e_1(A_n):=1p^n(A_n),e_2(A_n):=q^n(A_n),`$
and their exponents are given by

$`e_1^{}(\stackrel{}{A}):=\underset{n\mathrm{}}{lim\; inf}{\displaystyle \frac{1}{n}}\mathrm{log}e_1(A_n)`$
$`e_2^{}(\stackrel{}{A}):=\underset{n\mathrm{}}{lim\; inf}{\displaystyle \frac{1}{n}}\mathrm{log}e_2(A_n).`$
Chernoff  evaluated the exponent of the sum of the two errors as

$`\underset{\stackrel{}{A}}{sup}\underset{n\mathrm{}}{lim}{\displaystyle \frac{1}{n}}\mathrm{log}(e_1(A_n)+e_2(A_n))=\underset{\stackrel{}{A}}{sup}\mathrm{min}\{e_1^{}(\stackrel{}{A}),e_2^{}(\stackrel{}{A})\}=\underset{0<s<1}{sup}I^s(pq),`$ (21)
which is essential for our proof of Theorem 1. This bound is achieved by both of the likelihood tests $`\{\stackrel{}{\omega }_n\mathrm{\Omega }^n|p^n(\stackrel{}{\omega }_n)q^n(\stackrel{}{\omega }_n)\}`$ and $`\{\stackrel{}{\omega }_n\mathrm{\Omega }^n|p^n(\stackrel{}{\omega }_n)>q^n(\stackrel{}{\omega }_n)\}`$.

Hoeffding proved another formula for simple hypothesis testing :

$`\underset{\stackrel{}{A}:e_1^{}(\stackrel{}{A})r}{sup}e_2^{}(\stackrel{}{A})=\underset{0<s<1}{sup}{\displaystyle \frac{sr+I^s(pq)}{1s}}.`$ (22)
This formula is essential for our proof of Theorem 2.

Proof of Theorem 1:  Applying equation (21) to the two hypotheses $`p_{\theta ϵ}`$ and $`p_{\theta +ϵ}`$, we obtain

$`\underset{\theta ϵ\theta ^{}\theta +ϵ}{inf}\beta (\stackrel{}{T},\theta ^{},ϵ)\mathrm{min}\{\beta ^+(\stackrel{}{T},\theta ϵ,ϵ),\beta ^{}(\stackrel{}{T},\theta +ϵ,ϵ)\}\underset{0<s<1}{sup}I^s(p_{\theta ϵ}p_{\theta +ϵ}).`$
Taking the limit $`ϵ0`$, we have

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{g(ϵ)}}\underset{\stackrel{}{T}:WC}{sup}\underset{\theta ϵ\theta ^{}\theta +ϵ}{inf}\beta (\stackrel{}{T},\theta ^{},ϵ)2^\kappa \underset{0<s<1}{sup}I_{g,\theta }^s.`$ (23)
In the regular case, this method was used by Sievers .

Proof of Theorem 2: Hoeffding’s formula (22) yields

$`\underset{\theta (1\eta )ϵ\theta ^{}\theta +(1\eta )ϵ}{inf}\beta (\stackrel{}{T},\theta ^{},(1\eta )ϵ)`$ $`\beta ^{}(\stackrel{}{T},\theta +{\displaystyle \frac{1}{2}}ϵ,(1\eta )ϵ)`$
$`\underset{0<s<1}{sup}{\displaystyle \frac{s\beta ^+(\stackrel{}{T},\theta \frac{1}{2}ϵ,\eta ϵ)+I^s(p_{\theta \frac{1}{2}ϵ}p_{\theta +\frac{1}{2}ϵ})}{1s}}`$
$`\underset{0<s<1}{sup}{\displaystyle \frac{sinf_{\theta \eta ϵ\theta ^{}\theta +\eta ϵ}\beta (\stackrel{}{T},\theta ^{},\eta ϵ)+I^s(p_{\theta \frac{1}{2}ϵ}p_{\theta +\frac{1}{2}ϵ})}{1s}}.`$
The uniformity of (11) guarantees that

$`\alpha _2(\stackrel{}{T},\theta )(1\eta )^\kappa `$ $`\underset{0<s<1}{sup}{\displaystyle \frac{s\alpha _2(\stackrel{}{T},\theta )\eta ^\kappa +I_{g,\theta }^s}{1s}}.`$ (24)
From (24), we have

$`\alpha _2(\stackrel{}{T},\theta )sup\left\{\alpha \right|\alpha (\eta ^\kappa ,(1\eta )^\kappa )\left\{(x,y)\right|y\underset{0s1}{\mathrm{max}}{\displaystyle \frac{sx+I_g^s}{1s}},x,y0\},0\eta 1\}.`$ (25)
We define the set $`𝒞_1`$ and $`\alpha _0`$ as

$`𝒞_1`$ $`:=\left\{(x,y)\right|y\underset{0<t<1}{sup}{\displaystyle \frac{tx+I_{g,\theta }^t}{1t}}\}`$ (26)
$`\alpha _0`$ $`:=sup\{\alpha |(\alpha \eta ^\kappa ,\alpha (1\eta )^\kappa )int(𝒞_1),0\eta 1\}.`$ (27)
Note that the function $`sI_{g,\theta }^s`$ is concave. We define the convex function $`g(x)`$ and another set $`𝒞_2`$ as

$`g(x)`$ $`:=\{\begin{array}{cc}\alpha _0\left(1\left(\frac{x}{\alpha _0}\right)^{\frac{1}{\kappa }}\right)^\kappa & 0x\alpha _0\\ 0& x>\alpha _0\end{array}`$ (30)
$`𝒞_2`$ $`:=\{(x,y)|yg(x)\}.`$ (31)
Since inequality (25) guarantees

$`(\alpha _2(\stackrel{}{T},\theta )\eta ^\kappa ,\alpha _2(\stackrel{}{T},\theta )(1\eta )^\kappa )int(𝒞_1),0\eta 1,`$
the inequality

$`\alpha _2(\stackrel{}{T},\theta )\alpha _0`$ (32)
holds. Relations (27), (30), and (31) guarantee the relation

$`𝒞_1𝒞_2.`$ (33)
Applying Lemma 9, we have

$`\underset{(x,y)𝒞_1}{inf}(sx+(1s)y)=\underset{x0}{inf}\left(sx+(1s)\underset{0<t<1}{sup}{\displaystyle \frac{tx+I_{g,\theta }^t}{1t}}\right)`$
$`=`$ $`\underset{x>0}{inf}\underset{0<t<1}{sup}\left({\displaystyle \frac{(st)x+(1s)I_{g,\theta }^t}{1t}}\right)=I_{g,\theta }^s.`$ (34)
In the following, we divide our situation into three cases $`\kappa >1,\kappa =1,1>\kappa >0`$. When $`\kappa >1`$, relation (33) guarantees that

$`\alpha _0s(1s)\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{1\kappa }=\alpha _0\underset{0<\eta <1}{\mathrm{min}}\left(s\eta ^\kappa +(1s)(1\eta )^\kappa \right)`$
$`=`$ $`\underset{(x,y)𝒞_2}{inf}\left(sx+(1s)y\right)\underset{(x,y)𝒞_1}{inf}(sx+(1s)y)=I_{g,\theta }^s.`$
Therefore,

$`\alpha _2(\stackrel{}{T},\theta )\alpha _0{\displaystyle \frac{\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}}{s(1s)}}I_{g,\theta }^s,`$
which implies (16).

When $`\kappa =1`$, similarly, we can easily prove

$`{\displaystyle \frac{1}{2}}\alpha _2(\stackrel{}{T},\theta )=\alpha _2(\stackrel{}{T},\theta )\underset{0<\eta <1}{\mathrm{min}}\left({\displaystyle \frac{1}{2}}\eta +{\displaystyle \frac{1}{2}}(1\eta )\right)\underset{(x,y)𝒞_1}{inf}\left({\displaystyle \frac{1}{2}}x+{\displaystyle \frac{1}{2}}y\right)=I_{g,\theta }^{\frac{1}{2}}.`$
Finally, we consider the case where $`\kappa <1`$. Since function $`g`$ is concave on $`(0,\alpha _0)`$, there exists $`\eta _0(0,1)`$ such that

$`\alpha _0(\eta _0^\kappa ,(1\eta _0)^\kappa )𝒞_1\overline{𝒞_2^c}.`$ (35)
Since $`𝒞_1`$ and $`𝒞_2^c`$ are convex, there exists a real number $`s_0(0,1)`$ such that

$`\underset{(x,y)𝒞_2^c}{sup}\left(s_0x+(1s_0)y\right)=\alpha _0(s_0\eta _0^\kappa +(1s_0)(1\eta _0)^\kappa )=\underset{(x,y)𝒞_1}{inf}\left(s_0x+(1s_0)y\right).`$
In general, for any $`s(0,1)`$, using (34), we obtain

$`\alpha _0s(1s)\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{1\kappa }=\alpha _0\underset{0<\eta <1}{sup}\left(s\eta ^\kappa +(1s)(1\eta )^\kappa \right)=\underset{(x,y)𝒞_2^c}{sup}\left(sx+(1s)y\right)`$
$``$ $`\alpha _0\left(s\eta _0^\kappa +(1s)(1\eta _0)^\kappa \right)\underset{(x,y)𝒞_1}{inf}\left(sx+(1s)y\right)=I_{g,\theta }^s,`$
which lead to (16).

Proof of Theorem 3: It is trivial in the case of $`\kappa =1`$. When $`\kappa >1`$, it is also trivial because

$`\underset{0<s<1}{inf}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}2^\kappa I_{g,\theta }^{\frac{1}{2}}2^\kappa \underset{0s1}{sup}I_{g,\theta }^s.`$
Next, we consider the case $`\kappa <1`$. The inequality

$`\underset{0<s<1}{sup}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}2^\kappa \underset{0s1}{sup}I_{g,\theta }^s`$ (36)
follows from the two inequalities

$`I_{g,\theta }^s`$ $`\underset{0<s<1}{sup}I_{g,\theta }^s`$ (37)
$`{\displaystyle \frac{1}{s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}`$ $`2^\kappa .`$ (38)
We thus obtain (18). In the following, we prove that the equality of (18) implies (19) and (19) implies (20) and the equality of (18) in the case where $`\kappa <1`$. If we assume that the equality of (36) holds, the equalities of (37) and (38) hold at the same $`s`$. The equality of (38) holds if and only if $`s=\frac{1}{2}`$. Therefore,

$`I_{g,\theta }^{\frac{1}{2}}=\underset{0<s<1}{sup}I_{g,\theta }^s,`$
which is equivalent to (19). If we assume that (19) holds, inequality (38) guarantees that

$`\overline{\alpha }_2(\theta )2^\kappa I_{g,\theta }^{\frac{1}{2}}.`$ (39)
Substituting $`\frac{1}{2}`$ into $`s`$ at the left hand side (LHS) in the definition of $`\overline{\alpha }_2(\theta )`$, we obtain

$`\overline{\alpha }_2(\theta )2^\kappa I_{g,\theta }^{\frac{1}{2}}.`$
Thus, equation (20) holds. Combining (19) and (20), we obtain the equality of (18).

## 5 First exponential rates of useful estimators

In the following, we discuss the first exponential rate $`\beta (\stackrel{}{T},\theta ,ϵ)`$ of a useful estimator $`\stackrel{}{T}`$ for a location shift family $`\{f(x\theta )|\theta \}`$, where $`f`$ is a probability density function on $``$.

When the function $`d_ϵ(x):=f(x+ϵ)/f(xϵ)`$ is monotonically decreasing w.r.t. $`x`$ when both $`f(x+ϵ)`$ and $`f(xϵ)`$ are not zero, we can define the likelihood ratio estimator $`\stackrel{}{\stackrel{~}{\theta }_ϵ}:=\{\stackrel{~}{\theta }_{n,ϵ}(x_1,\mathrm{},x_n)\}`$, which depends on the constant $`ϵ>0`$, as shown by

$`\stackrel{~}{\theta }_{n,ϵ}:={\displaystyle \frac{1}{2}}\left(sup\{z|k(z)<0\}+inf\{z|k(z)>0\}\right),`$ (40)
where the monotonically decreasing function $`k(z)`$ is defined by

$`k(z):={\displaystyle \frac{1}{n}}{\displaystyle \underset{i=1}{\overset{n}{}}}\left(\mathrm{log}f(x_iz+ϵ)\mathrm{log}f(x_izϵ)\right).`$ (41)
Note that when $`\mathrm{log}f(x)`$ is concave, the above condition is satisfied. This definition is well defined although the monotonically decreasing function $`k(z)`$ is not continuous.

If the support of $`f`$ is $`(a,b)`$, we need to modify the definition as follows. In this case, we modify the estimator $`\stackrel{~}{\theta }_{n,ϵ}`$ by using the two estimators $`\overline{\theta }_n:=\mathrm{max}\{x_1,\mathrm{},x_n\}b`$ and $`\underset{¯}{\theta }_n:=\mathrm{min}\{x_1,\mathrm{},x_n\}a`$. When $`\underset{¯}{\theta }_n\overline{\theta }_n>2ϵ`$, the estimated value is defined by (40) in the interval $`(\underset{¯}{\theta }_nϵ,\overline{\theta }_n+ϵ)`$. When $`\underset{¯}{\theta }_n\overline{\theta }_n2ϵ`$, we define $`\stackrel{~}{\theta }_{n,ϵ}:=\frac{1}{2}\left(\underset{¯}{\theta }_n+\overline{\theta }_n\right)`$. Moreover, when the support of $`f`$ is a half line $`(0,\mathrm{})`$, the estimated value is defined by (40) in the half line $`(\underset{¯}{\theta }_nϵ,\mathrm{})`$.

We have the following lemma. (A regular version of this lemma is discussed by Huber , Sievers , and Fu .)

###### Lemma 1

When $`\mathrm{log}f(x)`$ is concave, the equation

$`\mathrm{min}\{\beta ^{}(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ,ϵ),\beta ^+(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ,ϵ)\}=\underset{0<s<1}{sup}I^s(f_{\theta ϵ}f_{\theta +ϵ}).`$ (42)
holds, where $`f_\theta (x):=f(x\theta )`$.

Therefore, the equality of inequality (10) holds in this case.

Proof:  Note that $`\beta ^+(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ,ϵ)=\beta ^+(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ϵ,ϵ)`$ and $`\beta ^{}(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ,ϵ)=\beta ^{}(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta +ϵ,ϵ)`$ because of the shift-invariance. From the concavity, the condition $`\stackrel{~}{\theta }_{ϵ,n}\theta `$ is equivalent to the condition $`sup\{z|k(z)<0\}\theta `$, which implies that $`k(\theta )0`$. Thus, we have

$`{\displaystyle \underset{i}{}}\mathrm{log}f(x_i\theta +ϵ)\mathrm{log}f(x_i\theta ϵ)0.`$
Conversely, the condition

$`{\displaystyle \underset{i}{}}\mathrm{log}f(x_i\theta +ϵ)\mathrm{log}f(x_i\theta ϵ)>0`$
implies that $`k(\theta )>0`$. Thus, we have $`sup\{z|k(z)<0\}\theta `$, which is equivalent to the condition $`\stackrel{~}{\theta }_{ϵ,n}\theta `$. Therefore, we have the relations

$`\{f_{\theta ϵ}^n(\stackrel{}{x}_n)>f_{\theta +ϵ}^n(\stackrel{}{x}_n)\}\{\stackrel{~}{\theta }_{n,ϵ}\theta \}\{f_{\theta ϵ}^n(\stackrel{}{x}_n)f_{\theta +ϵ}^n(\stackrel{}{x}_n)\}.`$
Similarly, we can prove

$`\{f_{\theta +ϵ}^n(\stackrel{}{x}_n)>f_{\theta ϵ}^n(\stackrel{}{x}_n)\}\{\stackrel{~}{\theta }_{n,ϵ}\theta \}\{f_{\theta +ϵ}^n(\stackrel{}{x}_n)f_{\theta ϵ}^n(\stackrel{}{x}_n)\}.`$
Applying (21), we can prove

$`\mathrm{min}\{\beta ^{}(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta +ϵ,ϵ),\beta ^+(\stackrel{}{\stackrel{~}{\theta }_ϵ},\theta ϵ,ϵ)\}=\underset{0<s<1}{sup}I^s(f_{\theta ϵ}f_{\theta +ϵ}),`$
which implies equation (42).

###### Corollary 1

When the function $`\mathrm{log}f(x)`$ is concave, the equation

$`\alpha _1(\theta )=\overline{\alpha }_1(\theta )`$
holds.

###### Lemma 2

When $`f(x)`$ is monotonically decreasing, the estimator $`\stackrel{}{\underset{¯}{\theta }_ϵ}:=\{\underset{¯}{\theta }_{ϵ,n}:=\underset{¯}{\theta }_nϵ\}`$ satisfies the relations

$`\beta ^+(\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)`$ $`=\underset{0<s<1}{sup}I^s(f_{\theta ϵ}f_{\theta +ϵ})`$
$`\beta ^{}(\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)`$ $`=\mathrm{}.`$ (43)
Thus, in this case, the equation

$`\alpha _1(\theta )=\overline{\alpha }_1(\theta )`$
holds.

Proof:  Since $`\underset{¯}{\theta }_n>\theta `$, we have $`\underset{¯}{\theta }_{ϵ,n}>\theta ϵ`$, which implies (43). If $`\underset{¯}{\theta }_{ϵ,n}\theta +ϵ`$, we have $`\underset{¯}{\theta }_n\theta +2ϵ`$. Thus, $`f(x_i(\theta +2ϵ))f(x_i\theta )`$ for any $`i=1,\mathrm{},n`$. Therefore,

$`f_{\theta +2ϵ}^n(\stackrel{}{x}_n)f_\theta ^n(\stackrel{}{x}_n).`$
Conversely, if $`f_{\theta +2ϵ}^n(\stackrel{}{x}_n)f_\theta ^n(\stackrel{}{x}_n)`$, we have $`\underset{¯}{\theta }_n\theta +2ϵ`$. Thus,

$`f_{\theta +2ϵ}^n\{\stackrel{}{x}_n|f_{\theta +2ϵ}^n(\stackrel{}{x}_n)<f_\theta ^n(\stackrel{}{x}_n)\}=f_{\theta +2ϵ}^n\{\stackrel{}{x}_n|\underset{¯}{\theta }_n<\theta +2ϵ\}=0.`$
Since the likelihood test $`\{\stackrel{}{x}_n|\underset{¯}{\theta }_n\theta +2ϵ\}`$ achieves the optimal rate (21), we have

$`lim{\displaystyle \frac{1}{n}}\mathrm{log}f_\theta ^n\{\stackrel{}{x}_n|f_{\theta +2ϵ}^n(\stackrel{}{x}_n)f_\theta ^n(\stackrel{}{x}_n)\}`$
$`=`$ $`lim{\displaystyle \frac{1}{n}}\mathrm{log}\left(f_\theta ^n\{\stackrel{}{x}_n|f_{\theta +2ϵ}^n(\stackrel{}{x}_n)f_\theta ^n(\stackrel{}{x}_n)\}+f_{\theta +2ϵ}^n\{\stackrel{}{x}_n|f_{\theta +2ϵ}^n(\stackrel{}{x}_n)<f_\theta ^n(\stackrel{}{x}_n)\}\right)`$
$`=`$ $`\underset{0<s<1}{sup}I^s(f_\theta f_{\theta +2ϵ})=\underset{0<s<1}{sup}I^s(f_{\theta ϵ}f_{\theta +ϵ}).`$
###### Lemma 3

When the function $`x\mathrm{log}f(x)`$ is concave, MLE $`\stackrel{}{\theta _{ML}}:=\{\theta _{ML,n}\}`$ satisfies that

$`\beta ^+(\stackrel{}{\theta _{ML}},\theta ,ϵ)`$ $`=\underset{t0}{sup}\mathrm{log}{\displaystyle _{a+ϵ}^b}\mathrm{exp}\left(t{\displaystyle \frac{f^{}(xϵ)}{f(xϵ)}}\right)f(x)𝑑x`$ (44)
$`=\underset{t0}{sup}\mathrm{log}{\displaystyle _a^{bϵ}}\mathrm{exp}\left(t{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(x+ϵ)𝑑x`$ (45)
$`\beta ^{}(\stackrel{}{\theta _{ML}},\theta ,ϵ)`$ $`=\underset{t0}{sup}\mathrm{log}{\displaystyle _a^{bϵ}}\mathrm{exp}\left(t{\displaystyle \frac{f^{}(x+ϵ)}{f(x+ϵ)}}\right)f(x)𝑑x`$ (46)
$`=\underset{t0}{sup}\mathrm{log}{\displaystyle _{a+ϵ}^b}\mathrm{exp}\left(t{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(xϵ)𝑑x.`$ (47)
This lemma is a special case of Fu’s result . For the reader’s convenience, we give its proof.

Proof:  Equations (45) and (47) are trivial. We prove (44). From the assumption that for any $`\stackrel{}{x}_n:=(x_1,\mathrm{},x_n)`$, the function $`\theta _{i=1}^n\mathrm{log}f(x_i\theta )`$ is concave on $`(\overline{\theta }(\stackrel{}{x}_n),\underset{¯}{\theta }(\stackrel{}{x}_n))`$, the function $`\theta _{i=1}^nl_\theta (x_i)`$ is monotonically decreasing, where $`l_\theta (x):=\frac{f^{}(x\theta )}{f(x\theta )}`$ on $`(\overline{\theta }(\stackrel{}{x}_n),\underset{¯}{\theta }(\stackrel{}{x}_n))`$. When $`\stackrel{}{x}_n`$ belongs to the support of $`f_\theta ^{}`$, the condition $`\theta _{ML,n}(\stackrel{}{x}_n)\theta ^{}`$ is equivalent to the condition

$`{\displaystyle \frac{1}{n}}{\displaystyle \underset{i=1}{\overset{n}{}}}l_\theta ^{}(x_i)0.`$
Denoting the conditional probability $`f\{A|xB\}`$ under the condition $`xB`$, we can evaluate

$`lim{\displaystyle \frac{1}{n}}\mathrm{log}f_\theta ^n\{\theta _{ML,n}\theta +ϵ\}`$ $`=lim{\displaystyle \frac{1}{n}}\mathrm{log}f_\theta ^n\{\theta _{ML,n}\theta +ϵ|\stackrel{}{x}_n(a+ϵ,b+ϵ)^n\}{\displaystyle \frac{1}{n}}\mathrm{log}f_\theta ^n(a+ϵ,b+ϵ)^n`$
$`=lim{\displaystyle \frac{1}{n}}\mathrm{log}f_{\theta ,ϵ}^n\left\{{\displaystyle \frac{1}{n}}{\displaystyle \underset{i=1}{\overset{n}{}}}l_{\theta +ϵ}(x_i)0\right\}\mathrm{log}{\displaystyle _{a+ϵ}^b}f(x)𝑑x,`$ (48)
where the probability density function whose support is $`(a+ϵ,b)`$ is defined by

$`f_{\theta ,ϵ}(x):={\displaystyle \frac{f(x)}{_{a+ϵ}^bf(x)𝑑x}}.`$
Chernoff’s theorem (Theorem 3.1 in Bahadur ) guarantees that

$`lim{\displaystyle \frac{1}{n}}\mathrm{log}f_{\theta ,ϵ}^n\{{\displaystyle \frac{1}{n}}{\displaystyle \underset{i=1}{\overset{n}{}}}l_{\theta +ϵ}(x_i)0\}`$ $`=\underset{t0}{sup}\mathrm{log}{\displaystyle _{a+ϵ}^b}\mathrm{exp}\left(tl_{\theta +ϵ}(x)\right)f_{\theta ,ϵ}(x)𝑑x`$
$`=\underset{t0}{sup}\mathrm{log}{\displaystyle _{a+ϵ}^b}\mathrm{exp}\left(t{\displaystyle \frac{f^{}(x\theta )}{f(x\theta )}}\right)f(x)𝑑x+\mathrm{log}{\displaystyle _{a+ϵ}^b}f(x)𝑑x.`$ (49)
Combining (48) and (49), we obtain (44). Similarly, We can prove (46).

###### Lemma 4

When $`f(x)`$ is monotonically decreasing, the MLE $`\theta _{ML,n}`$ equals the estimator $`\underset{¯}{\theta }_n`$.

Proof:  For any data $`\stackrel{}{x}_n:=(x_1,\mathrm{},x_n)`$, if $`\theta <\underset{¯}{\theta }_n(\stackrel{}{x}_n))`$, $`f(x_1\theta )\mathrm{}f(x_n\theta )=0`$. Conversely, if $`\theta >\underset{¯}{\theta }_n(\stackrel{}{x}_n))`$, the obtained $`f(x_i\theta )f(x_i\underset{¯}{\theta }_n(\stackrel{}{x}_n))`$. Thus, $`\underset{¯}{\theta }_n`$ is the MLE.

###### Lemma 5

Let $`f`$ be a density function whose support is the interval $`(a,b)`$. The estimators $`\underset{¯}{\overset{}{\theta }}`$ and $`\stackrel{}{\overline{\theta }}`$ satisfy

$`\beta ^+(\underset{¯}{\overset{}{\theta }},\theta ,ϵ)`$ $`=\mathrm{log}\left({\displaystyle _a^{bϵ}}f(x)𝑑x\right),\beta ^{}(\underset{¯}{\overset{}{\theta }},\theta ,ϵ)=\mathrm{}`$ (50)
$`\beta ^+(\stackrel{}{\overline{\theta }},\theta ,ϵ)`$ $`=\mathrm{},\beta ^{}(\stackrel{}{\overline{\theta }},\theta ,ϵ)=\mathrm{log}\left({\displaystyle _{a+ϵ}^b}f(x)𝑑x\right)`$ (51)
We can use the convex combination $`\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )}:=\{\stackrel{ˇ}{\theta }(\lambda )_n:=\lambda \underset{¯}{\theta }_n+(1\lambda )\overline{\theta }_n\}`$ with the ratio $`\lambda :1\lambda `$, where $`0<\lambda <1`$. It satisfies that

$`\beta ^+(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)`$ $`=\mathrm{log}\left({\displaystyle _a^{b\frac{ϵ}{1\lambda }}}f(x)𝑑x\right)`$ (52)
$`\beta ^{}(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)`$ $`=\mathrm{log}\left({\displaystyle _{a+\frac{ϵ}{\lambda }}^b}f(x)𝑑x\right).`$ (53)
Proof:  Define $`\overline{\omega }_n:=\mathrm{max}\{\omega _1,\mathrm{},\omega _n\},\underset{¯}{\omega }_n:=\mathrm{min}\{\omega _1,\mathrm{},\omega _n\}`$. Since the estimators $`\underset{¯}{\overset{}{\theta }},\stackrel{}{\overline{\theta }}`$, and $`\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )}`$ are covariant for location shift, we may discuss only the case that $`\theta =0`$. From the relation $`\underset{¯}{\theta }_n>\theta >\theta ϵ`$, we obtain the second equation of (50). Its joint probability density function $`f_n(\overline{\omega }_n,\underset{¯}{\omega }_n)`$ is given by

$`f_n(\overline{\omega }_n,\underset{¯}{\omega }_n):=\{\begin{array}{cc}n(n1)\left(_{\underset{¯}{\omega }_n}^{\overline{\omega }_n}f(x)𝑑x\right)^{n2}f(\underset{¯}{\omega }_n)f(\overline{\omega }_n)\hfill & \overline{\omega }_n\underset{¯}{\omega }_n\hfill \\ 0\hfill & \overline{\omega }_n<\underset{¯}{\omega }_n.\hfill \end{array}`$
Defining

$`g(\underset{¯}{\omega }_n,\overline{\omega }_n):=\{\begin{array}{cc}_{\underset{¯}{\omega }_n}^{\overline{\omega }_n}f(x)𝑑x\hfill & \overline{\omega }_n\underset{¯}{\omega }_n\hfill \\ 0\hfill & \overline{\omega }_n<\underset{¯}{\omega }_n,\hfill \end{array}`$
we have

$`p_\theta ^n(\stackrel{ˇ}{\theta }(\lambda )_n\theta ϵ)={\displaystyle _{\stackrel{ˇ}{\theta }(\lambda )(\underset{¯}{\omega }_n,\overline{\omega }_n)ϵ}}n(n1)g(\underset{¯}{\omega }_n,\overline{\omega }_n)^nf(\underset{¯}{\omega }_n)f(\overline{\omega }_n)𝑑\underset{¯}{\omega }_n𝑑\overline{\omega }_n.`$ (54)
From the continuity of $`f(\underset{¯}{\omega }_n),f(\overline{\omega }_n)`$ and $`g(\underset{¯}{\omega }_n,\overline{\omega }_n)`$, the equations

$`\underset{n\mathrm{}}{lim}{\displaystyle \frac{1}{n}}\mathrm{log}p_\theta ^n(\stackrel{ˇ}{\theta }(\lambda )_n\theta ϵ)`$ $`=\underset{n\mathrm{}}{lim}{\displaystyle \frac{1}{n}}\underset{\stackrel{ˇ}{\theta }(\lambda )(\underset{¯}{\omega }_n,\overline{\omega }_n)ϵ}{sup}\mathrm{log}g(\underset{¯}{\omega }_n,\overline{\omega }_n)`$
$`=\mathrm{log}\left({\displaystyle _a^{b\frac{ϵ}{1\lambda }}}f(x)𝑑x\right)`$
hold. This implies the first equation of (50) and (52). In addition, we can similarly show the same for (51) and (53).

## 6 Two bounds in location shift family

We discuss a location shift family generated by a probability density function (pdf) whose support is an interval $`(a,b)`$. Moreover, we assume that the pdf $`f`$ is $`C^1`$ continuous and satisfies that

$`f(x)A_1(xa)^{\kappa _11},`$ $`xa+0`$
$`f(x)A_2(bx)^{\kappa _21},`$ $`xb0,`$
where $`\kappa _1,\kappa _2>0`$, as for the uniform and beta distributions.

When its support is a half line $`(0,\mathrm{})`$ as for the gamma distribution and Weibull distribution, our situation results in the above case where $`A_2=0`$ if $`f`$ is $`C^3`$ continuous and $`lim_x\mathrm{}|\frac{d}{dx}\mathrm{log}f(x)|<\mathrm{}`$. Also, when $`\kappa _1>\kappa _2`$, our situation results in the above case where $`A_2=0`$.

In the following, in this setting, we calculate two upper bounds $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$, and derive a necessary and sufficient condition for coincidence of the two upper bounds. In addition, in some cases, we calculate the two bounds $`\alpha _1(\theta )`$ and $`\alpha _2(\theta )`$.

### 6.1 Semi-regular case

As was proven in , when $`\kappa _1,\kappa _2>2`$, the relation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(f_\theta f_{\theta +ϵ})}{ϵ^2}}={\displaystyle \frac{s(1s)}{2}}J_f`$ (55)
holds, where this convergence is uniform for $`s`$ and $`J_f`$ is defined by

$`J_f:={\displaystyle _a^b}\left({\displaystyle \frac{df(x)}{dx}}\right)^2f(x)^1𝑑x.`$
###### Proposition 1

When $`g(x)=x^2`$, we obtain $`\kappa =2`$ and the relation

$`\overline{\alpha }_1(\theta )=\alpha _1(\theta )=\overline{\alpha }_2(\theta )=\alpha _2(\theta )={\displaystyle \frac{1}{2}}J_\theta .`$ (56)
Proof:  Using (55), we have

$`\overline{\alpha }_1(\theta )`$ $`=4\underset{0s1}{\mathrm{max}}I_{g,\theta }^s={\displaystyle \frac{1}{2}}J_\theta `$
$`\overline{\alpha }_2(\theta )`$ $`=\underset{0s1}{\mathrm{min}}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}={\displaystyle \frac{1}{2}}J_\theta .`$
When the function $`\mathrm{log}f(x)`$ is concave, using Lemma 3 we can evaluate

$`\beta ^+(\stackrel{}{\theta _{ML}},\theta ,ϵ)`$ $`\mathrm{log}{\displaystyle _a^{bϵ}}\mathrm{exp}\left(ϵ{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(x+ϵ)𝑑x`$
$`\mathrm{log}\left({\displaystyle _a^{bϵ}}\left(1ϵ{\displaystyle \frac{f^{}(x)}{f(x)}}+{\displaystyle \frac{ϵ^2}{2}}\left({\displaystyle \frac{f^{}(x)}{f(x)}}\right)^2\right)\left(f(x)+ϵf^{}(x)+{\displaystyle \frac{ϵ^2}{2}}f^{\prime \prime }(x)\right)𝑑x+o(ϵ^2)\right)`$
$`\mathrm{log}\left({\displaystyle _a^{bϵ}}f(x){\displaystyle \frac{ϵ^2}{2}}\left({\displaystyle \frac{f^{}(x)^2}{f(x)}}+f^{\prime \prime }(x)\right)dx+o(ϵ^2)\right)`$
$`\mathrm{log}\left(1{\displaystyle _{bϵ}^b}f(x)𝑑x{\displaystyle \frac{ϵ^2}{2}}{\displaystyle _a^{bϵ}}{\displaystyle \frac{f^{}(x)^2}{f(x)}}𝑑x+o(ϵ^2)\right)`$
$`\mathrm{log}\left(1{\displaystyle \frac{ϵ^2}{2}}\left({\displaystyle _a^{bϵ}}{\displaystyle \frac{f^{}(x)^2}{f(x)}}𝑑xf^{}(bϵ)\right)+o(ϵ^2)\right).`$
Thus, we obtain

$`\underset{ϵ0}{lim}{\displaystyle \frac{1}{ϵ^2}}\beta ^+(\stackrel{}{\theta _{ML}},\theta ,ϵ){\displaystyle \frac{1}{2}}\underset{ϵ0}{lim}\left({\displaystyle _a^{bϵ}}{\displaystyle \frac{f^{}(x)^2}{f(x)}}𝑑xf^{}(bϵ)\right)={\displaystyle \frac{1}{2}}J_f.`$
Similarly, we can prove that

$`\underset{ϵ0}{lim}{\displaystyle \frac{1}{ϵ^2}}\beta ^{}(\stackrel{}{\theta _{ML}},\theta ,ϵ){\displaystyle \frac{1}{2}}J_f.`$
Since $`\alpha _2(\stackrel{}{\theta _{ML}},\theta )\frac{1}{2}J_f`$, we have

$`\alpha _2(\stackrel{}{\theta _{ML}},\theta )={\displaystyle \frac{1}{2}}J_f,`$
which implies (56).

### 6.2 The case that $`\kappa _1=\kappa _2=1`$

As was proven elsewhere  , when $`\kappa _1=\kappa _2=1`$, the equation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(f_\theta f_{\theta +ϵ})}{ϵ}}=A_1s+A_2(1s)`$
holds, where this convergence is uniform for $`s(0,1)`$. Letting $`g(x)=|x|`$, we have $`\kappa =1`$.

###### Proposition 2

The relations

$`\alpha _1(\theta )`$ $`=\overline{\alpha }_1(\theta )=2\underset{0<s<1}{sup}I_{g,\theta }^s=2\mathrm{max}\{A_1,A_2\}`$ (57)
$`\alpha _2(\theta )`$ $`=\overline{\alpha }_2(\theta )=2I_{g,\theta }^{\frac{1}{2}}=A_1+A_2`$ (58)
hold. Therefore, $`\alpha _1(\theta )=\alpha _2(\theta )`$ if and only if $`A_1=A_2`$.

Proof:  The third equations of (57) and (58) follow from the formula $`I_{g,\theta }^s=A_1s+A_2(1s)`$. In the following, we prove the first equations of (57) and (58). By using Lemma 5, the equations

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ}}\beta ^+(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)={\displaystyle \frac{A_1}{\lambda }},\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ}}\beta ^{}(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)={\displaystyle \frac{A_2}{1\lambda }}`$
hold. Letting $`\lambda _0:=A_1/(A_1+A_2)`$, we have

$`\alpha _2(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda _0)},\theta )=A_1+A_2,`$
which implies the first equation of (58).

Next, we prove the first equation of (57) in the case where $`A_1A_2`$. The estimator $`\stackrel{}{\underset{¯}{\theta }_ϵ}:=\{\underset{¯}{\theta }_{ϵ,n}\}`$ satisfies

$`\beta ^+(\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)=\mathrm{log}{\displaystyle _{a+2ϵ}^b}f(x)𝑑x,\beta ^{}(\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)=\mathrm{}.`$
Therefore,

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ}}\beta (\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)=2A_1,`$
which implies the first equation of (57). When $`A_2A_1`$, we can similarly prove it.

### 6.3 The case that $`\kappa _1=\kappa _2=2`$

As was proven by , when $`\kappa _1=\kappa _2=2`$ the equation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(f_\theta f_{\theta +ϵ})}{ϵ^2\mathrm{log}|ϵ|}}={\displaystyle \frac{(A_1+A_2)s(1s)}{2}}`$ (59)
holds, where this convergence is uniform for $`s(0,1)`$. Letting $`g(x)=x^2\mathrm{log}x`$, we have $`\kappa =2`$. Using (59), we have

$`\overline{\alpha }_1(\theta )`$ $`=4\underset{0<s<1}{sup}I_{g,\theta }^s={\displaystyle \frac{A_1+A_2}{2}}`$ (60)
$`\overline{\alpha }_2(\theta )`$ $`=\underset{0<s<1}{inf}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}={\displaystyle \frac{A_1+A_2}{2}}.`$
###### Proposition 3

When the function $`x\mathrm{log}f(x)`$ is concave, we have

$`\alpha _1(\theta )=\overline{\alpha }_1(\theta )=\alpha _2(\theta )=\overline{\alpha }_2(\theta )={\displaystyle \frac{A_1+A_2}{2}}.`$ (61)
Proof:  When $`a+\delta <x<b\delta `$, we can approximate that

$`\mathrm{exp}\left(ϵ{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(x+ϵ)f(x)+{\displaystyle \frac{ϵ^2}{2}}\left(f^{\prime \prime }(x){\displaystyle \frac{(f^{}(x))^2}{f(x)}}\right).`$ (62)
Therefore, from Lemma 3, we can evaluate

$`\beta ^+(\stackrel{}{\theta _{ML}},\theta ,ϵ)`$
$`\mathrm{log}\left({\displaystyle _a^{bϵ}}\mathrm{exp}\left(ϵ{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(x+ϵ)𝑑x\right)`$
$`=\mathrm{log}\left({\displaystyle _{a+\delta }^{b\delta }}\mathrm{exp}\left(ϵ{\displaystyle \frac{f^{}(x)}{f(x)}}\right)f(x+ϵ)𝑑x+{\displaystyle _0^\delta }\mathrm{exp}\left(ϵ{\displaystyle \frac{A_1}{A_1x}}\right)A_1(x+ϵ)𝑑x+{\displaystyle _ϵ^\delta }\mathrm{exp}\left(+ϵ{\displaystyle \frac{A_2}{A_2x}}\right)A_2(xϵ)𝑑x\right)`$
$`\mathrm{log}({\displaystyle _{a+\delta }^{b\delta }}f(x)dx+{\displaystyle _a^{a+\delta }}f(x)dx+{\displaystyle _{b\delta }^b}f(x)dx`$
$`+A_1{\displaystyle _0^\delta }(\mathrm{exp}(ϵ{\displaystyle \frac{1}{x}})(x+ϵ)x)dx+A_2{\displaystyle _ϵ^\delta }(\mathrm{exp}(+ϵ{\displaystyle \frac{1}{x}})(xϵ)x)dx+o(ϵ^2\mathrm{log}ϵ))`$
$`\mathrm{log}\left(1+{\displaystyle \frac{A_1+A_2}{2}}ϵ^2\mathrm{log}ϵ+o(ϵ^2\mathrm{log}ϵ)\right)`$ (63)
$`{\displaystyle \frac{A_1+A_2}{2}}ϵ^2\mathrm{log}ϵ+o(ϵ^2\mathrm{log}ϵ),`$
where the relation (63) follows from Lemma 11. Thus, we have

$`\alpha _2(\stackrel{}{\theta _{ML}},\theta )={\displaystyle \frac{A_1+A_2}{2}}.`$ (64)
Using (60) and (64), we obtain (61).

### 6.4 The case that $`1<\kappa _1=\kappa _2<2`$

As was proven by Hayashi , when $`1<\kappa _1=\kappa _2<2`$, the equation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^{\kappa _1}}}I^s(f_\theta f_{\theta +ϵ})`$
$`=`$ $`{\displaystyle \frac{A_1s(1s(\kappa _11))B(s+\kappa _1(1s),2\kappa _1)}{\kappa _1}}+{\displaystyle \frac{A_2(1s)(1(1s)(\kappa _11))B(1s+\kappa _1s,2\kappa _1)}{\kappa _1}}`$
holds, where this convergence is uniform for $`s(0,1)`$, and $`B(x,y)`$ is a beta function. Letting $`g(x)=|x|^{\kappa _1}`$, we have $`\kappa =\kappa _1`$.

$`\overline{\alpha }_1(\theta )=2^\kappa \underset{0<s<1}{sup}I_{g,\theta }^s`$
$`=2^\kappa \underset{0s1}{\mathrm{max}}{\displaystyle \frac{A_1s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )+A_2(1s)(1(1s)(\kappa 1))B(1s+\kappa s,2\kappa )}{\kappa }}`$
$`\overline{\alpha }_2(\theta )=\underset{0<s<1}{inf}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}`$
$`=\underset{0<s<1}{inf}[{\displaystyle \frac{A_1s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )+A_2(1s)(1(1s)(\kappa 1))B(1s+\kappa s,2\kappa )}{\kappa s(1s)}}`$
$`(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}})^{\kappa 1}]`$
$`{\displaystyle \frac{1}{\kappa }}(A_1+A_2)2^{\kappa 2}(1\kappa )B({\displaystyle \frac{1+\kappa }{2}},2\kappa ).`$
From Lemma 5, we have

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^{\kappa _1}}}\beta ^+(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)`$ $`=A_1{\displaystyle \frac{1}{\kappa _1\lambda ^{\kappa _1}}}`$ (65)
$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^{\kappa _2}}}\beta ^{}(\stackrel{}{\stackrel{ˇ}{\theta }(\lambda )},\theta ,ϵ)`$ $`=A_2{\displaystyle \frac{1}{\kappa _2(1\lambda )^{\kappa _2}}}.`$ (66)
Thus, the estimator $`\stackrel{ˇ}{\theta }(\lambda )`$ achieves the optimal order. However, it does not achieve the optimal coefficient $`\overline{\alpha }_2(\theta )`$.

###### Proposition 4

If, and only if, $`A_1=A_2`$, the equality

$`\overline{\alpha }_1(\theta )=\overline{\alpha }_2(\theta )`$
holds. In this case,

$`\overline{\alpha }_1(\theta )=\overline{\alpha }_2(\theta )={\displaystyle \frac{A_12^{\kappa 1}(3\kappa )B(\frac{1+\kappa }{2},2\kappa )}{\kappa }}.`$ (67)
Proof:  When $`A_1A_2`$, (79) and (80) of Lemma 6 guarantee that

$`{\displaystyle \frac{d}{ds}}I_{g,\theta }^s|_{s=\frac{1}{2}}=(A_1A_2){\displaystyle \frac{(\kappa 1)(3\kappa )}{4}}\pi \mathrm{tan}{\displaystyle \frac{2\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},2\kappa )s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )0.`$ (68)
From the concavity and the continuity of the maximized function, we have $`\overline{\alpha }_1(\theta )>2^\kappa I_{g,\theta }^{\frac{1}{2}}`$. When $`A_1=A_2`$, we have $`I_{g,\theta }^s=I_{g,\theta }^{1s}`$. The relations

$`\overline{\alpha }_1(\theta )=2^\kappa I_{g,\theta }^{\frac{1}{2}}={\displaystyle \frac{A_12^{\kappa 1}(3\kappa )B(\frac{1+\kappa }{2},2\kappa )}{\kappa }}`$ (69)
follow from the concavity. Since the minimums

$`\underset{0s1}{\mathrm{min}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)`$
and

$`\underset{0s1}{\mathrm{min}}\left({\displaystyle \frac{(1s(\kappa 1))B(s+\kappa (1s),2\kappa )}{1s}}+{\displaystyle \frac{(1(1s)(\kappa 1))B((1s)+\kappa s,2\kappa )}{s}}\right)`$
are achieved at the same point $`s=\frac{1}{2}`$ (See Lemma 8), the relation

$`\underset{0s1}{\mathrm{min}}(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}})^{\kappa 1}({\displaystyle \frac{(1s(\kappa 1))B(s+\kappa (1s),2\kappa )}{1s}}+{\displaystyle \frac{(1(1s)(\kappa 1))B((1s)+\kappa s,2\kappa )}{s}})`$
$`=`$ $`2^{\kappa 1}(3\kappa )B({\displaystyle \frac{1+\kappa }{2}},2\kappa )`$ (70)
holds. Thus, equation (67) follows from (69) and (70).

Next, we consider the case $`A_2=0`$.

###### Proposition 5

When $`1<\kappa <2t_0`$, the relations

$`\alpha _1(\theta )=\overline{\alpha }_1(\theta )=A_1{\displaystyle \frac{2^\kappa }{\kappa }}`$ (71)
hold, where the real number $`t_0(0,\frac{1}{2})`$ is uniquely defined by (see Lemma 7)

$`2t_0+t_0(1t_0)\left(\psi (1+t_0)\psi (1)\right)=1,`$ (72)
where $`\psi (x)`$ is the D-psi function defined by $`\psi (x):=\frac{d}{dx}\mathrm{log}\mathrm{\Gamma }(x)`$.

The number $`t_0`$ is enumerated by $`t_00.432646`$, as is checked by the following graph.

Proof:  In this case, since the function $`ss(1s(\kappa 1))B(s+\kappa (1s),2\kappa )`$ is concave, Lemma 7 guarantees that

$`{\displaystyle \frac{d}{ds}}s(1s(\kappa 1))B(s+\kappa (1s),2\kappa ))`$ $`{\displaystyle \frac{d}{ds}}s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )|_{s=1}`$
$`=(32\kappa )+(2\kappa )(\kappa 1)(\psi (3\kappa )\psi (1))0`$
for $`s(0,1)`$. Thus,

$`\overline{\alpha }_1(\theta )`$ $`={\displaystyle \frac{A_12^\kappa }{\kappa }}\underset{0s1}{\mathrm{max}}s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )`$
$`={\displaystyle \frac{A_12^\kappa }{\kappa }}1(11(\kappa 1))B(1+\kappa (11),2\kappa )={\displaystyle \frac{A_12^\kappa }{\kappa }}.`$
As in subsection 6.2, we can prove that

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^\kappa }}\beta (\stackrel{}{\underset{¯}{\theta }_ϵ},\theta ,ϵ)={\displaystyle \frac{A_12^\kappa }{\kappa }},`$
which implies (71).

### 6.5 The case that $`0<\kappa _1=\kappa _2<1`$

As was proven by Hayashi , when $`0<\kappa _1=\kappa _2<1`$, the equation

$`\underset{ϵ+0}{lim}{\displaystyle \frac{I^s(f_\theta f_{\theta +ϵ})}{ϵ^{\kappa _1}}}={\displaystyle \frac{1\kappa _1}{\kappa _1}}(A_1sB(s+\kappa _1(1s),1\kappa _1)+A_2(1s)B(1s+\kappa _1s,1\kappa _1))`$
holds, where this convergence is uniform for $`s(0,1)`$. Letting $`g(x)=x_1^\kappa `$, we have $`\kappa =\kappa _1`$.

$`\overline{\alpha }_1(\theta )=2^\kappa \underset{0s1}{\mathrm{max}}I_{g,\theta }^s`$
$`=2^\kappa \underset{0s1}{\mathrm{max}}(1\kappa ){\displaystyle \frac{A_1sB(s+\kappa (1s),1\kappa )+A_2(1s)B(1s+\kappa s,1\kappa )}{\kappa }}`$
$`\overline{\alpha }_2(\theta )=\underset{0<s<1}{sup}{\displaystyle \frac{I_{g,\theta }^s}{s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}`$
$`=\underset{0<s<1}{sup}(1\kappa ){\displaystyle \frac{A_1sB(s+\kappa (1s),1\kappa )+A_2(1s)B(1s+\kappa s,1\kappa )}{\kappa s(1s)}}\left(s^{\frac{1}{\kappa 1}}+(1s)^{\frac{1}{\kappa 1}}\right)^{\kappa 1}.`$
###### Proposition 6

If, and only if, $`A_1=A_2`$, the equality

$`\overline{\alpha }_1(\theta )=\overline{\alpha }_2(\theta )`$ (73)
holds. In this case, the equation

$`\overline{\alpha }_1(\theta )=\overline{\alpha }_2(\theta )={\displaystyle \frac{1}{\kappa }}A_12^\kappa \left(1\kappa \right)B({\displaystyle \frac{1+\kappa }{2}},1\kappa )`$ (74)
holds.

Proof:  Using (81) and (82) of Lemma 6, we obtain

$`{\displaystyle \frac{d}{ds}}I_{g,\theta }^s|_{s=\frac{1}{2}}=(A_1A_2){\displaystyle \frac{1\kappa }{2}}\pi \mathrm{cot}{\displaystyle \frac{1\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},1\kappa ).`$
Since $`\frac{1\kappa }{2}\pi \mathrm{cot}\frac{1\kappa }{2}\pi B(\frac{1+\kappa }{2},1\kappa )>0`$, Theorem 3 yields this sufficient and necessary condition for (73). Equation (19) implies (74).

However, since the function $`x(\kappa 1)\mathrm{log}x`$ is convex on $`(0,\mathrm{})`$, the function $`x\mathrm{log}f(x)`$ is not concave on $`(a,b)`$. There does not exist an example in which Lemma 1 can be applied. Thus, it is an open problem whether there exists an example such that

$`\overline{\alpha }_1(\theta )=\alpha _1(\theta )`$
in this case, except for the case $`A_1A_2=0`$.

###### Proposition 7

When $`A_2=0`$, $`\alpha _1(\theta )`$ and $`\alpha _2(\theta )`$ are calculated as

$`\alpha _1(\theta )`$ $`=\overline{\alpha }_1(\theta )={\displaystyle \frac{A_12^\kappa }{\kappa }}`$ (75)
$`\alpha _2(\theta )`$ $`=\overline{\alpha }_2(\theta )={\displaystyle \frac{A_1}{\kappa }}.`$ (76)
Proof:  Since the function $`ssB(s+\kappa (1s),1\kappa )`$ is concave, we have

$`{\displaystyle \frac{d}{ds}}sB(s+\kappa (1s),1\kappa )`$ $`{\displaystyle \frac{d}{ds}}sB(s+\kappa (1s),1\kappa )|_{s=1}`$
$`=1+(1\kappa )(\psi (1)\psi (2\kappa ))`$
$`1+(1\kappa )(\psi (1)\psi (2))`$ (77)
$`=1(1\kappa )=\kappa >0,`$ (78)
where inequality (77) holds because $`\psi (x)`$ is monotonically increasing in $`x(0,\mathrm{})`$, and the first equation of (78) follows from the formula $`\psi (x+1)=\psi (x)+\frac{1}{x}`$. Thus,

$`\overline{\alpha }_1(\theta )`$ $`={\displaystyle \frac{A_12^\kappa (1\kappa )}{\kappa }}\underset{0s1}{\mathrm{max}}sB(s+\kappa (1s),1\kappa )`$
$`={\displaystyle \frac{A_12^\kappa (1\kappa )}{\kappa }}B(1,1\kappa )={\displaystyle \frac{A_12^\kappa }{\kappa }}.`$
Since, as in subsection 6.4, we can check that the estimators $`\{\underset{¯}{\theta }_ϵ\}_{ϵ>0}`$ achieve the bound $`\frac{A_12^\kappa }{\kappa }`$, equations (75) hold.

The other upper bound $`\overline{\alpha }_2(\theta )`$ is calculated as

$`\overline{\alpha }_2(\theta )={\displaystyle \frac{A_1(1\kappa )}{\kappa }}\underset{0s1}{\mathrm{max}}B(s+\kappa (1s),1\kappa )\left(\left({\displaystyle \frac{1s}{s}}\right)^{\frac{1}{1\kappa }}+1\right)^{(1\kappa )}.`$
Note that the beta function $`B(x,y)`$ is monotonically decreasing for $`x,y>0`$. Since both $`\mathrm{max}_{0s1}B(s+\kappa (1s),1\kappa )`$ and $`\mathrm{max}_{0s1}(\left(\frac{1s}{s}\right)^{\frac{1}{1\kappa }}+1)^{(1\kappa )}`$ are achieved at the same point, $`s=1`$, we have

$`\overline{\alpha }_2(\theta )={\displaystyle \frac{A_1(1\kappa )}{\kappa }}B(1,1\kappa )={\displaystyle \frac{A_1}{\kappa }}.`$
This bound is achieved by the estimator $`\underset{¯}{\overset{}{\theta }}`$ because

$`\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^\kappa }}\beta (\underset{¯}{\overset{}{\theta }},\theta ,ϵ)=\underset{ϵ+0}{lim}{\displaystyle \frac{1}{ϵ^\kappa }}\mathrm{log}{\displaystyle _{a+ϵ}^b}f(x)𝑑x={\displaystyle \frac{A_1}{\kappa }}.`$
Therefore, we have (76).

## 7 Conclusion

We have discussed large deviation theories under a more general setting. The two quantities $`\overline{\alpha }_1(\theta )`$ and $`\overline{\alpha }_2(\theta )`$ do not necessarily coincide. In a non-regular case, it is clear that the order of limits is crucial. In the future, such phenomena deserve study from another viewpoint.

Nagaoka  initiated a discussion of two kinds of large deviation bounds, as in this paper, in a quantum setting, and Hayashi  discussed these in more depth. The two kinds of large deviation bounds do not necessarily coincide in a quantum setting. However, the reason for this difference in a quantum setting differs from that for a non-regular setting. Gaining an understanding of these differences from a unified viewpoint remains a goal for the future.

## Appendix A Lemmas concerning the beta and D-psi functions

In this section, we prove some formulas concerning the beta and D-psi functions used in Section 6.

###### Lemma 6

When $`1<\kappa <2`$, we have

$`{\displaystyle \frac{d}{ds}}s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )|_{s=1/2}`$ $`={\displaystyle \frac{(\kappa 1)(3\kappa )}{4}}\pi \mathrm{tan}{\displaystyle \frac{2\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},2\kappa )>0`$ (79)
$`{\displaystyle \frac{d}{ds}}(1s)(1(1s)(\kappa 1))B((1s)+\kappa s,2\kappa )|_{s=1/2}`$ $`={\displaystyle \frac{(\kappa 1)(3\kappa )}{4}}\pi \mathrm{tan}{\displaystyle \frac{2\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},2\kappa )<0`$ (80)
When $`0<\kappa <1`$, the equations

$`{\displaystyle \frac{d}{ds}}sB(s+\kappa (1s),1\kappa )|_{s=1/2}`$ $`={\displaystyle \frac{1\kappa }{2}}\pi \mathrm{cot}{\displaystyle \frac{1\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},1\kappa )>0`$ (81)
$`{\displaystyle \frac{d}{ds}}(1s)B((1s)+\kappa s,1\kappa )|_{s=1/2}`$ $`={\displaystyle \frac{1\kappa }{2}}\pi \mathrm{cot}{\displaystyle \frac{1\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},1\kappa )<0`$ (82)
hold.

Proof:  Using the function $`\psi (x):=\frac{d}{dx}\mathrm{log}\mathrm{\Gamma }(x)`$, we can calculate

$`{\displaystyle \frac{d}{ds}}s(1s(\kappa 1))B(s+\kappa (1s),2\kappa )|_{s=1/2}`$
$`=\left((2\kappa )+{\displaystyle \frac{(3\kappa )(1\kappa )}{4}}\left(\psi ({\displaystyle \frac{1+\kappa }{2}})\psi ({\displaystyle \frac{5\kappa }{2}})\right)\right)B({\displaystyle \frac{1+\kappa }{2}},2\kappa )`$
$`=\left((2\kappa )+{\displaystyle \frac{(3\kappa )(1\kappa )}{4}}\left(\psi ({\displaystyle \frac{\kappa 1}{2}})+{\displaystyle \frac{2}{\kappa 1}}\psi ({\displaystyle \frac{3\kappa }{2}}){\displaystyle \frac{2}{3\kappa }}\right)\right)B({\displaystyle \frac{1+\kappa }{2}},2\kappa )`$ (83)
$`=\left((2\kappa )+{\displaystyle \frac{(3\kappa )(1\kappa )}{4}}\left(\pi \mathrm{cot}\pi {\displaystyle \frac{3\kappa }{2}}+{\displaystyle \frac{84\kappa }{(\kappa 1)(3\kappa )}}\right)\right)B({\displaystyle \frac{1+\kappa }{2}},2\kappa )`$ (84)
$`={\displaystyle \frac{(\kappa 1)(3\kappa )}{4}}\pi \mathrm{tan}{\displaystyle \frac{2\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},2\kappa )`$ (85)
where we use the formula $`\psi (x+1)=\frac{1}{x}+\psi (x)`$ in (83), the formula $`\psi (1x)\psi (x)=\pi \mathrm{cot}\pi x`$ in (84), and the formula $`\mathrm{cot}(\frac{\pi }{2}+x)=\mathrm{tan}x`$ in (85). We obtain (79). Similarly, we can prove (80).

Next, we prove (81). We can calculate

$`{\displaystyle \frac{d}{ds}}sB(s+\kappa (1s),1\kappa )|_{s=1/2}`$
$`=\left(1+{\displaystyle \frac{1\kappa }{2}}\left(\psi ({\displaystyle \frac{1+\kappa }{2}})\psi ({\displaystyle \frac{3\kappa }{2}})\right)\right)B({\displaystyle \frac{1+\kappa }{2}},1\kappa )`$
$`=\left(1+{\displaystyle \frac{1\kappa }{2}}\left(\psi ({\displaystyle \frac{1+\kappa }{2}})\psi ({\displaystyle \frac{3\kappa }{2}}){\displaystyle \frac{2}{1\kappa }}\right)\right)B({\displaystyle \frac{1+\kappa }{2}},1\kappa )`$ (86)
$`={\displaystyle \frac{1\kappa }{2}}\pi \mathrm{cot}{\displaystyle \frac{1\kappa }{2}}\pi B({\displaystyle \frac{1+\kappa }{2}},1\kappa ),`$ (87)
where (86) follows from the formula $`\psi (x+1)=\frac{1}{x}+\psi (x)`$ and (87) follows from the formula $`\psi (1x)\psi (x)=\pi \mathrm{cot}\pi x`$. Similarly, we obtain (82).

###### Lemma 7

Assume that $`1<\kappa <2`$. There uniquely exists $`t_0(0,\frac{1}{2})`$ satisfying (72), and the inequality

$`(32\kappa )+(2\kappa )(\kappa 1)(\psi (3\kappa )\psi (1))0`$ (88)
holds if, and only if, $`1<\kappa 2t_0`$.

Proof:  In the following, this lemma is proven by replacing $`\kappa `$ with $`2t`$. Define the function $`h(t):=2t1+t(1t)(\psi (1+t)\psi (1))`$. When $`t<\frac{1}{2}`$, we have

$`h^{}(t)=2+(12t)(\psi (1+t)\psi (1))+(t2t^2)\psi ^{}(1+t)>0`$
because $`\psi ^{}(x)0`$ for $`x>0`$. Therefore, $`h(t)`$ is strictly monotonically increasing in $`(0,\frac{1}{2})`$. Since $`h(0)=1<0,h(\frac{1}{2})=\frac{1}{4}\left(\psi (\frac{3}{2})\psi (1)\right)>0`$, there uniquely exists the number $`t_0(0,\frac{1}{2})`$ satisfying (72). Also, in this case, the inequality $`h(t)0`$ holds if $`tt_0`$.

Next, we consider case $`t\frac{1}{2}`$. Given the relations,

$`2t10,t(1t)0,\psi (1+t)\psi (1)0,`$
$`h(t)0`$.

###### Lemma 8

The minimum

$`\underset{0s1}{\mathrm{min}}\left({\displaystyle \frac{(1s(\kappa 1))B(s+\kappa (1s),2\kappa )}{1s}}+{\displaystyle \frac{(1(1s)(\kappa 1))B((1s)+\kappa s,2\kappa )}{s}}\right)`$
is attained at $`s=\frac{1}{2}`$.

Proof:  Since the minimized function is invariant for the replacement $`s1s`$, it is sufficient to show its concavity. Since $`\frac{d^2}{dx^2}e^{h(x)}=(h^{\prime \prime }(x)+(h(^{}x))^2)e^{h(x)}`$, we can show its concavity by proving the concavity of the function $`s\mathrm{log}\frac{1s(\kappa 1)}{1s}B(s+\kappa (1s),2\kappa )`$. We can evaluate

$`{\displaystyle \frac{d^2}{ds^2}}\mathrm{log}B(s+\kappa (1s),2\kappa )=(1\kappa )^2\left(\psi ^{}(s+\kappa (1s))\psi ^{}(s+\kappa (1s)+2\kappa )\right)>0`$
because $`\psi ^{}(x)`$ is monotonically decreasing for $`x>0`$. Also, we have

$`{\displaystyle \frac{d^2}{ds^2}}\mathrm{log}{\displaystyle \frac{1s(\kappa 1)}{1s}}={\displaystyle \frac{(2\kappa )(\kappa 2(\kappa 1)s)}{(1+\kappa s+(\kappa 1)s^2)^2}}>0`$
because $`\kappa >2(\kappa 1)`$. The proof is now complete.

## Appendix B Concave function

###### Lemma 9

When a concave function $`f0`$ is defined in $`(0,1)`$,

$`\underset{x0}{inf}sx+(1s)\underset{0<t<1}{sup}{\displaystyle \frac{tx+f(t)}{1t}}=\underset{x>0}{inf}\underset{0<t<1}{sup}{\displaystyle \frac{(st)x+(1s)f(t)}{1t}}=f(s).`$
Proof:  Substituting $`s`$ into $`t`$ we have

$`f(s)\underset{0<t<1}{sup}{\displaystyle \frac{(st)x+(1s)f(t)}{1t}}.`$
Taking the infimum $`inf_{x>0}`$, we obtain

$`f(s)\underset{x>0}{inf}\underset{0<t<1}{sup}{\displaystyle \frac{(st)x+(1s)f(t)}{1t}}.`$
Next, we proceed to the opposite inequality. From the concavity of $`f`$, we can define the upper derivative $`\overline{f^{}}`$ and the lower derivative $`\underset{¯}{f^{}}`$ as

$`\overline{f^{}}(s):=\underset{ϵto+0}{lim}{\displaystyle \frac{f(s)f(sϵ)}{ϵ}},\underset{¯}{f^{}}(s):=\underset{ϵto+0}{lim}{\displaystyle \frac{f(s+ϵ)f(s)}{ϵ}}`$
Since the concavity guarantees that

$`{\displaystyle \frac{f(s+ϵ)f(s)}{ϵ}}\underset{¯}{f^{}}(s)\overline{f^{}}(s){\displaystyle \frac{f(s)f(sϵ)}{ϵ}},ϵ>0,`$
we obtain

$`{\displaystyle \frac{(ss)(f(s)+(1s)\overline{f^{}}(s))+(1s)f(s)}{1s}}{\displaystyle \frac{(st)(f(s)+(1s)\overline{f^{}}(s))+(1s)f(t)}{1t}}`$
$`=`$ $`{\displaystyle \frac{1s}{1t}}\left(f(s)f(t)+(ts)\overline{f^{}}(s)\right)0,s,t(0,1).`$
Therefore,

$`f(s)\underset{0<t<1}{sup}{\displaystyle \frac{(st)(f(s)+(1s)\overline{f^{}}(s))+(1s)f(t)}{1t}}\underset{x>0}{inf}\underset{0<t<1}{sup}{\displaystyle \frac{(st)(f(s)+(1s)\overline{f^{}}(s))+(1s)f(t)}{1t}}.`$
The proof is now complete.

## Appendix C Other lemmas

###### Lemma 10

When $`g`$ is strictly monotonically decreasing and continuous and satisfies that $`g(0)=0`$, there exists $`\kappa >0`$ such that

$`x^\kappa =\underset{ϵ+0}{lim}{\displaystyle \frac{g(xϵ)}{g(ϵ)}},x>0.`$ (89)
Proof:  Let $`h(x)`$ be the RHS of (89). Since

$`\underset{ϵ+0}{lim}{\displaystyle \frac{g(xyϵ)}{g(ϵ)}}=\underset{ϵ+0}{lim}{\displaystyle \frac{g(xyϵ)}{g(yϵ)}}\underset{ϵ+0}{lim}{\displaystyle \frac{g(yϵ)}{g(ϵ)}},`$
$`h(xy)=h(x)h(y)`$. Thus, there exists $`\kappa >0`$ satisfying (89).

###### Lemma 11

For any $`\delta >0`$, we have

$`\underset{ϵ0}{lim}{\displaystyle \frac{1}{ϵ^2\mathrm{log}ϵ}}{\displaystyle _0^\delta }\mathrm{exp}\left(ϵ{\displaystyle \frac{1}{x}}\right)(x+ϵ)xdx`$ $`={\displaystyle \frac{1}{2}}`$ (90)
$`\underset{ϵ0}{lim}{\displaystyle \frac{1}{ϵ^2\mathrm{log}ϵ}}{\displaystyle _ϵ^\delta }\mathrm{exp}\left(+ϵ{\displaystyle \frac{1}{x}}\right)(xϵ)xdx`$ $`={\displaystyle \frac{1}{2}}.`$ (91)
Proof:  we can calculate

$`{\displaystyle _ϵ^\delta }\mathrm{exp}\left(+ϵ{\displaystyle \frac{1}{x}}\right)(xϵ)xdx={\displaystyle _ϵ^\delta }{\displaystyle \underset{n=1}{\overset{\mathrm{}}{}}}{\displaystyle \frac{nϵ^{n+1}}{(n+1)!x^n}}dx={\displaystyle \frac{ϵ^2}{2}}(\mathrm{log}\delta \mathrm{log}ϵ)+{\displaystyle \underset{n=2}{\overset{\mathrm{}}{}}}{\displaystyle \frac{n(n1)}{(n+1)!}}\left({\displaystyle \frac{ϵ^{n+1}}{\delta ^{n1}}}ϵ^2\right).`$
Since $`\frac{ϵ^2}{ϵ^2\mathrm{log}ϵ}0`$, we obtain (91). Similarly, we can calculate

$`{\displaystyle _ϵ^\delta }\mathrm{exp}\left(ϵ{\displaystyle \frac{1}{x}}\right)(x+ϵ)xdx={\displaystyle \frac{ϵ^2}{2}}(\mathrm{log}\delta \mathrm{log}ϵ)+{\displaystyle \underset{n=2}{\overset{\mathrm{}}{}}}{\displaystyle \frac{n(n1)(1)^n}{(n+1)!}}\left({\displaystyle \frac{ϵ^{n+1}}{\delta ^{n1}}}ϵ^2\right).`$
Since

$`0{\displaystyle _0^ϵ}\mathrm{exp}\left(ϵ{\displaystyle \frac{1}{x}}\right)(x+ϵ)𝑑x`$ $`{\displaystyle _0^ϵ}(x+ϵ)𝑑x={\displaystyle \frac{3}{2}}ϵ^2`$
$`{\displaystyle _0^ϵ}x𝑑x`$ $`={\displaystyle \frac{1}{2}}ϵ^2,`$
we obtain (90).

###### Lemma 12

For $`0<s<\frac{1}{2}`$, the inequalities

$`2sI^{\frac{1}{2}}(pq)I^s(pq)2(1s)I^{\frac{1}{2}}(pq)`$
hold.

Proof:  Since $`\left(\frac{1}{2s}\right)^1+\left(\frac{1}{12s}\right)^1=1`$, the Hölder inequality guarantees that

$`{\displaystyle _\mathrm{\Omega }}p^s(\omega )q^{1s}(\omega )𝑑\omega `$ $`={\displaystyle _\mathrm{\Omega }}\left(p^s(\omega )q^s(\omega )\right)\left(q^{12s}(\omega )\right)𝑑\omega `$
$`\left({\displaystyle _\mathrm{\Omega }}\left(p^s(\omega )q^s(\omega )\right)^{\frac{1}{2s}}𝑑\omega \right)^{2s}\left({\displaystyle _\mathrm{\Omega }}\left(q^{12s}(\omega )\right)^{\frac{1}{12s}}𝑑\omega \right)^{12s}`$
$`=\left({\displaystyle _\mathrm{\Omega }}p^{\frac{1}{2}}(\omega )q^{\frac{1}{2}}(\omega )𝑑\omega \right)^{2s}.`$
Thus, we obtain

$`I^s(pq)2sI^{\frac{1}{2}}(pq).`$
Similarly, since $`\left(2(1s)\right)^1+\left(\frac{2(1s)}{12s}\right)^1=1`$, we can apply the Hölder inequality as

$`{\displaystyle _\mathrm{\Omega }}p^{\frac{1}{2}}(\omega )q^{\frac{1}{2}}(\omega )𝑑\omega `$ $`={\displaystyle _\mathrm{\Omega }}\left(p^s(\omega )q^{1s}(\omega )\right)^{\frac{1}{2(1s)}}p^{\frac{12s}{2(1s)}}(\omega )𝑑\omega `$
$`\left({\displaystyle _\mathrm{\Omega }}\left(p^s(\omega )q^{1s}(\omega )\right)^{\frac{1}{2(1s)}2(1s)}𝑑\omega \right)^{\frac{1}{2(1s)}}\left({\displaystyle _\mathrm{\Omega }}p^{\frac{12s}{2(1s)}\frac{2(1s)}{12s}}(\omega )𝑑\omega \right)^{\frac{12s}{2(1s)}}`$
$`=\left({\displaystyle _\mathrm{\Omega }}p^s(\omega )q^{1s}(\omega )𝑑\omega \right)^{\frac{1}{2(1s)}},`$
which implies that

$`I^{\frac{1}{2}}(pq){\displaystyle \frac{1}{2(1s)}}I^s(pq).`$