A Mathematical Approach to the Theory of Finite Automata

1. Deterministic Finite Automaton (DFA)

Definition 1.

A deterministic finite automaton denoted by $DFA$ is a 5-tuple,

$M=(Q,\Sigma ,\delta ,q_0,F),$ where

(i) Q is a finite set of states;

(ii) Σ is a finite alphabet;

(iii) $\delta :Q\times \Sigma ⟶Q$ is the transition function;

(iv) $q_0\in Q$ is the start state; and

(v) $F\subset Q$ is the set of accept states.

Let $w=w_1{w}_2{w}_3,\dots ,w_n$ be a string over Σ where each $w_i\in \Sigma$ and $n\ge 1$.

M accepts w if and only if $\exists r_0,r_1,r_2,\dots ,r_n\in Q$ s.t. the following conditions are satisfied:

(a) $r_0=q_0$;

(b) $\delta (r_i,w_{i+1})=r_{i+1}$ for $i=0,1,2,\dots ,n-1$; and

(c) $r_n\in F$

For $n=0,w=ϵ.$ Only conditions (a) and (c) are applicable and they become $r_0=q_0$ and $r_0\in F$. We therefore define M to accept ϵ if the start state is also an accept state.

On the other hand, since there is no ϵ-movement in a $DFA$, the only way the $DFA$ can accept an empty string is to accept it at the start state.

Accordingly, M accepts ϵ if and only if the start state is also an accept state.

If we write $r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}$ instead of $\delta (r_i,w_{i+1})=r_{i+1}$ for $i=0,1,2,\dots ,n-1,$ then conditions (a), (b) and (c) can be written as follows:

$q_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n,r_n\in F$

We say M recognizes language A if $A=\{w\in {\Sigma }^{*}\mid M$ accepts $w\}$ and it is written as $L\left(M\right)=A$

Definition 2.

A language is called regular if it is recognized by a $DFA$.

Definition 3.

For any language L,

$L^0=\left\{ϵ\right\},L^1=L,L^2=LL,\dots ,L^{m+1}=L^{m}L$ for $m>0$.

$L^{*}=L^0\cup L^1\cup L^2\cup \dots ={\displaystyle \bigcup _{k=0}^{\infty }}{L}^k$

$=\{w$|$w=w_1{w}_2{w}_3\dots w_n;w_i\in L$ for $1\le i\le n;n\ge 1\}\cup \{ϵ\}$

Definition 4.

Inductive Transition Function

Let $M=(Q,\Sigma ,\delta ,q_0,F)$ be a $DFA$.

$\widehat{\delta }:Q\times {\Sigma }^{*}⟶Q$ s.t.

(i) $\widehat{\delta }(q,ϵ)=q\forall q\in Q$

(ii) $\widehat{\delta }(q,wa)=\delta (\widehat{\delta }(q,w),a)\forall a\in \Sigma ,w\in {\Sigma }^{*},q\in Q$

Definition 5.

$\forall p,q\in Q,w\in {\Sigma }^{*},p\stackrel{w,\widehat{\delta }}{⟶}q\stackrel{def}{⟺}q=\widehat{\delta }(p,w)$

Proposition 1.

$\widehat{\delta }(q,a)=\delta (q,a)$$\forall q\in Q,a\in \Sigma$

$<Proof>$

$\begin{array}{ll}\widehat{\delta }(q,a)=\widehat{\delta }(q,ϵa)& \\ =\delta (\widehat{\delta }(q,ϵ),a)& \left(Definition1.4\left(ii\right)\right)\\ =\delta (q,a)& \left(Definition1.4\left(i\right)\right)\end{array}$

Theorem 1.

$\left(\mathit{DFA}\mathit{Acceptance}\right)$

For any $DFA$, $M=(Q,\Sigma ,\delta ,q_0,F)$

$\widehat{\delta }(q_0,w)\in F⟺M$ accepts $w\forall w\in {\Sigma }^{*}$

$<Proof>$

Claim: If $w=w_1{w}_2\dots w_n$ where $n\ge 0$ and

$q_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n,$ then $\widehat{\delta }(r_0,w)=r_n$

This Claim can be proved by induction on n.

For $n=0,w=ϵ$ and the computation becomes $q_0=r_0$.

$\widehat{\delta }(q_0,w)=\widehat{\delta }(q_0,ϵ)$

$=q_0$   (By Definition 1.4(i))

$=r_0$

Therefore, the statement is true for $n=0$.

Assume the statement is true for $n=k$, where $k\ge 0$.

That is, $\widehat{\delta }(r_0,w_1{w}_2\dots w_k)=r_k$

$\begin{array}{cc}\widehat{\delta }(r_0,w_1{w}_2\dots w_k{w}_{k+1})=\delta (\widehat{\delta }(r_0,w_1{w}_2\dots w_k),w_{k+1})\hfill & \hfill \mathit{\left(}\mathit{Definition}\mathit{1.4}\mathit{\right(}ii\mathit{\left)}\mathit{\right)}\\ =\delta (r_k,w_{k+1})\hfill & \hfill \mathit{\left(}\mathit{Induction}\mathit{Hypothesis}\mathit{\right)}\\ =r_{k+1}\hfill & \hfill \mathit{\left(}\mathit{Definition}\mathit{of}{r}_{i+1}\mathit{\right)}\end{array}$

Therefore, the statement is true for $n=k+1$.

If M accepts $w=w_1{w}_2\dots w_n,$ where $w_i\in \Sigma$ for $1\le i\le n$ and $n\ge 1$ or $(w=ϵ$ and $n=0)$

$\exists r_0,r_1,r_2,\dots r_n\in Q$ st

$q_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n,$$r_n\in F$

By Claim, $\widehat{\delta }(r_0,w_1{w}_2\dots w_n)=r_n$

Therefore, $\widehat{\delta }(q_0,w)=r_n$ ($r_0=q_0;w=w_1{w}_2\dots w_n$)

Since $r_n\in F,$

$\widehat{\delta }(q_0,w)\in F$

Therefore, M accepts $w⟹\widehat{\delta }(q_0,w)\in F$

Conversely, if $\widehat{\delta }(q_0,w)\in F$

$\widehat{\delta }(q_0,w_1{w}_2\dots w_n)\in F$

Take $r_0=q_0$

$r_{i+1}=\delta (r_i,w_{i+1})\forall i=0,1,2,\dots n-1$

$q_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n$

By Claim, $\widehat{\delta }(r_0,w_1{w}_2\dots w_n)=r_n$

Since $\widehat{\delta }(q_0,w_1{w}_2\dots w_n)\in F,r_n\in F$.

Therefore, M accepts w.

Therefore, $\widehat{\delta }(q_0,w)\in F⟹M$ accepts w.

Therefore, $\widehat{\delta }(q_0,w)\in F⟺M$ accepts w.

This completes the proof.

Theorem 2.

For any $DFAs,M$ and $M^{\prime }$ where

$M=(Q,\Sigma ,\delta ,q_0,F)$

$M^{\prime }=(Q,\Sigma ,{\delta }^{\prime },q_0,F^{\prime })$

$\forall q\in Q,a\in \Sigma ,w\in {\Sigma }^{*}$

${\delta }^{\prime }(q,a)=\delta (q,a)⟹\widehat{{\delta }^{\prime }}(q,w)=\widehat{\delta }(q,w)$

$<Proof>$

The proof is by induction on $\left|w\right|\ge 0$.

For $\left|w\right|=0,w=ϵ$.

By Definition 1.4(i),

$\widehat{\delta }(q,ϵ)=q$ and $\widehat{{\delta }^{\prime }}(q,ϵ)=q$

Therefore, $\widehat{\delta }(q,ϵ)=\widehat{{\delta }^{\prime }}(q,ϵ)$

Assume the statement is true for $\left|w\right|=k\ge 0$.

$\begin{array}{cc}\widehat{\delta }(q,wa)=\delta (\widehat{\delta }(q,w),a)\hfill & \\ ={\delta }^{\prime }(\widehat{\delta }(q,w),a)\hfill & \mathit{(}{\delta }^{\prime }(q,a)=\delta (q,a)\mathit{)}\\ ={\delta }^{\prime }(\widehat{{\delta }^{\prime }}(q,w),a)\hfill & \mathit{\left(}\mathit{Induction}\mathit{Hypothesis}\mathit{\right)}\\ =\widehat{{\delta }^{\prime }}(q,wa)\hfill & \mathit{\left(}\mathit{Definition}\mathit{1.4}\mathit{\right(}ii\mathit{\left)}\mathit{\right)}\end{array}$

The statement is also true for $\left|w\right|=k+1$

2. Nondeterministic Finite Automaton (NFA)

Definition 6.

A nondeterministic finite automaton ($NFA$) is a 5-tuple,

$N=(Q,\Sigma ,\delta ,q_0,F)$, where

(i) Q is a finite set of states;

(ii) Σ is a finite alphabet;

(iii) $\delta :Q\times {\Sigma }_{ϵ}⟶P\left(Q\right)$ is the transition function, where

  ${\Sigma }_{ϵ}=\Sigma \cup \left\{ϵ\right\},P\left(Q\right)=$ the power set of $Q=\{S\mid S\subset Q\}$.

(iv) $q_0\in Q$ is the start state; and

(v) $F\subset Q$ is the set of accept states.

Let $w=w_1{w}_2{w}_3\dots w_m$ where $w_i\in {\Sigma }_{ϵ}$ for $1\le i\le m$ and $m\ge 1$.

N accepts w if and only if $\exists r_0,r_1,r_2,\dots ,r_m\in Q$ s.t. the following conditions are satisfied:

(a) $r_0\in \left\{q_0\right\}$

(b) $r_{i+1}\in \delta (r_i,w_{i+1})$ for $i=0,1,2,\dots ,m-1$

(c) $r_m\in F$

For $m=0,w=ϵ.$

Only conditions (a) and (c) are applicable and they become $r_0=q_0$ and $r_0\in F$.

We therefore define N to accept ϵ if the start state is also an accept state.

If we write $r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}$ instead of $r_{i+1}\in \delta (r_i,w_{i+1})$ for $i=0,1,2,\dots ,m-1$, then conditions (a), (b) and (c) can be written as follows:

$q_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_i\stackrel{w_{i+1},\delta }{⟶}{r}_{i+1}\dots r_{m-1}\stackrel{w_m,\delta }{⟶}{r}_m,r_m\in F$.

Note that when $m=0,$ this computation becomes $q_0=r_0$ and $r_0\in F.$

Definition 7.

(Inductive Transition Function)

Let $N=(Q,\Sigma ,\delta ,q_0,F)$ be an $NFA$.

$\widehat{\delta }:P\left(Q\right)\times {\Sigma }_{ϵ}^{*}⟶P\left(Q\right)$ such that

(i) $\widehat{\delta }(A,ϵ)=A\forall A\in P\left(Q\right)$

(ii) $\widehat{\delta }(A,wa)={\displaystyle \bigcup _{q\in \widehat{\delta }(A,w)}}\delta (q,a)\forall a\in {\Sigma }_{ϵ},w\in {\Sigma }_{ϵ}^{*},A\in P\left(Q\right)$.

Definition 8.

$\forall p,q\in Q,w\in {\Sigma }_{ϵ}^{*},p\stackrel{w,\widehat{\delta }}{⟶}q\stackrel{def}{⟺}q\in \widehat{\delta }(\left\{p\right\},w)$.

Proposition 2.

If $N=(Q,\Sigma ,\delta ,q_0,F)$ is an $NFA$, then

$\forall a\in {\Sigma }_{ϵ},p\in Q,\widehat{\delta }(\left\{p\right\},a)=\delta (p,a).$

$<Proof>$

$\widehat{\delta }(\left\{p\right\},a)=\widehat{\delta }(\left\{p\right\},ϵa)$

$={\displaystyle \bigcup _{q\in \widehat{\delta }(\left\{p\right\},ϵ)}}\delta (q,a)$ (Definition 1.10 (ii))

$={\displaystyle \bigcup _{q\in \left\{p\right\}}}\delta (q,a)$ (Definition 1.10 (i))

$=\delta (p,a)$

Proposition 3.

If $N=(Q,\Sigma ,\delta ,s_0,F)$ is an $NFA$,

$\forall w\in {\Sigma }_{ϵ}^{*}$ where $w=w_1{w}_2{w}_3\dots w_n$ ; $w_i\in {\Sigma }_{ϵ}$ for $1\le i\le n$ and $n\ge 1$ or $w=ϵ$ for $n=0.$

$(\exists r_0,r_1,r_2,\dots ,r_n\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n)⟺r_n\in \widehat{\delta }(\left\{s_0\right\},w)$

$<Proof>$

This proposition can be proved by induction on n.

Let $P\left(n\right)$ denote the statement:

$(\exists r_0,r_1,r_2,\dots ,r_n\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n)$; and

$Q\left(n\right)$ denote the statement: $r_n\in \widehat{\delta }(\left\{s_0\right\},w)$.

For $n=0,w=ϵ$.

$P\left(0\right)⟺(\exists r_0\in Q$ s.t. $s_0=r_0)$

$⟺r_0\in \left\{s_0\right\}$

$⟺r_0\in \widehat{\delta }(\left\{s_0\right\},ϵ)$ (Definition 1.10(i))

$⟺r_0\in \widehat{\delta }(\left\{s_0\right\},w)$ ($w=ϵ$)

$⟺Q\left(0\right)$

Assume $P\left(k\right)⟺Q\left(k\right)$ for any $k\ge 0$.

$P(k+1)⟺(\exists r_0,r_1,r_2,\dots r_k,r_{k+1}\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{k-1}\stackrel{w_k,\delta }{⟶}{r}_k\stackrel{w_{k+1},\delta }{⟶}{r}_{k+1})$

$P(k+1)⟹P\left(k\right)$ (From computation path of $P(k+1)$)

$⟹Q\left(k\right)$ (Induction Hypothesis)

$⟹r_k\in \widehat{\delta }(\left\{s_0\right\},w)$ where $w_1{w}_2{w}_3\dots w_k=w$ (Definition of $Q\left(k\right)$)

Since

$\widehat{\delta }(\left\{s_0\right\},w{w}_{k+1})={\displaystyle \bigcup _{q\in \widehat{\delta }(\left\{s_0\right\},w)}}\delta (q,w_{k+1})$(Definition 1.10(ii)

and $r_k\in \widehat{\delta }(\left\{s_0\right\},w)$,

$\delta (r_k,w_{k+1})\subset \widehat{\delta }(\left\{s_0\right\},w{w}_{k+1})$

$r_k\stackrel{w_{k+1},\delta }{⟶}{r}_{k+1}$ (From computation path of $P(k+1)$)

Therefore, $r_{k+1}\in \delta (r_k,w_{k+1})\subset \widehat{\delta }(\left\{s_0\right\},w{w}_{k+1})$

Therefore, $r_{k+1}\in \widehat{\delta }(\left\{s_0\right\},w{w}_{k+1})$

Therefore, $P(k+1)⟹Q(k+1)$.

Conversely,

$Q(k+1)⟹r_{k+1}\in \widehat{\delta }(\left\{s_0\right\},w{w}_{k+1})$

$⟹r_{k+1}\in {\displaystyle \bigcup _{q\in \widehat{\delta }(\left\{s_0\right\},w)}}\delta (q,w_{k+1})$

$r_{k+1}\in \delta (r_k,w_{k+1})$ for some $r_k\in \widehat{\delta }(\left\{s_0\right\},w)$

$r_k\in \widehat{\delta }(\left\{s_0\right\},w)⟹Q\left(k\right)$

$⟹P\left(k\right)$ (Induction Hypothesis)

$⟹(\exists r_0,r_1,r_2,\dots r_k\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{k-1}\stackrel{w_k,\delta }{⟶}{r}_k)$

$r_{k+1}\in \delta (r_k,w_{k+1})⟹r_k\stackrel{w_{k+1},\delta }{⟶}{r}_{k+1}$

Combining the two computation paths,

$s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{k-1}\stackrel{w_k,\delta }{⟶}{r}_k\stackrel{w_{k+1},\delta }{⟶}{r}_{k+1}$

Therefore, $Q(k+1)⟹P(k+1)$ and the proof is complete.

Proposition 4.

$\forall x,y\in {\Sigma }_{ϵ}^{*}$&$A\in P\left(Q\right)$, $\widehat{\delta }(A,xy)=\widehat{\delta }(\widehat{\delta }(A,x),y)$

$<Proof>$

The proof is by induction on $n=\left|y\right|$.

Let $T\left(n\right)$ denote the statement corresponding to $n=0,1,2,\dots$

For $\left|y\right|=0$, $y=ϵ$.

$\widehat{\delta }(A,xϵ)=\widehat{\delta }(A,x)$

$=\widehat{\delta }(\widehat{\delta }(A,x),ϵ)$ (Definition 1.10(i))

$T\left(0\right)$ is true.

Assume $T\left(k\right)$ is true for $\left|y\right|=k\ge 0$.

That is $\widehat{\delta }(A,xy)=\widehat{\delta }(\widehat{\delta }(A,x),y)$ for $\left|y\right|=k\ge 0$

For any $a\in {\Sigma }_{ϵ}$, $y\in {\Sigma }_{ϵ}^{*}$, $\left|y\right|=k$

LHS of $T(k+1)=\widehat{\delta }(A,xya)$

$={\displaystyle \bigcup _{q\in \widehat{\delta }(A,xy)}}\delta (q,a)$ (By Definition 1.10(ii))

$={\displaystyle \bigcup _{q\in \widehat{\delta }(\widehat{\delta }(A,x),y)}}\delta (q,a)$ (By Induction Hypothesis)

$=\widehat{\delta }(\widehat{\delta }(A,x),ya)$ (By Definition 1.10(ii))

$=$ RHS of $T(k+1)$

Therefore, $T\left(k\right)⟹T(k+1)$.

Proposition 5.

$\forall A_i\subset Q,x\in {\Sigma }_{ϵ}^{*},i=1,2,\dots n,n\in N,\widehat{\delta }\left({\displaystyle \bigcup _{i=1}^n}{A}_i,x\right)={\displaystyle \bigcup _{i=1}^n}\widehat{\delta }(A_i,x)$

$<Proof>$

The proof is by induction on $\left|x\right|$.

For $\left|x\right|=0$, $x=ϵ$.

$\widehat{\delta }\left({\displaystyle \bigcup _{i=1}^n}{A}_i,ϵ\right)={\displaystyle \bigcup _{i=1}^n}{A}_i$ (Definition 1.10(i))

$={\displaystyle \bigcup _{i=1}^n}\widehat{\delta }(A_i,ϵ)$ (Definition 1.10(i))

Claim: $\forall n\in N$, sets $A_i$ and $S_x$

${\displaystyle \bigcup _{x\in {\cup }_{i=1}^n{A}_i}}{S}_x={\displaystyle \bigcup _{i=1}^n}\left({\displaystyle \bigcup _{x\in A_i}}{S}_x\right)$

<Proof of Claim>

LHS $={\displaystyle \bigcup _{x\in A_1\cup A_2\cup \dots A_n}}{S}_x$

$=\left({\displaystyle \bigcup _{x\in A_1}}{S}_x\right)\cup \left({\displaystyle \bigcup _{x\in A_2}}{S}_x\right)\cup \dots \cup \left({\displaystyle \bigcup _{x\in A_n}}{S}_x\right)$

$={\displaystyle \bigcup _{i=1}^n}\left({\displaystyle \bigcup _{x\in A_i}}{S}_x\right)$

$=$ RHS

Assume the statement is true for $\left|x\right|=k$ for $k\ge 0$.

$\forall a\in {\Sigma }_{ϵ}$, $\left|xa\right|=k+1$.

$\widehat{\delta }\left({\displaystyle \bigcup _{i=1}^n}{A}_i,xa\right)={\displaystyle \bigcup _{p\in \widehat{\delta }(\left({\cup }_{i=1}^n{A}_i\right),x)}}\delta (p,a)$ (Definition 1.10(ii))

$={\displaystyle \bigcup _{p\in {\cup }_{i=1}^n\widehat{\delta }(A_i,x)}}\delta (p,a)$ (Induction Hypothesis)

$={\displaystyle \bigcup _{i=1}^n}\left({\displaystyle \bigcup _{p\in \widehat{\delta }(A_i,x)}}\delta (p,a)\right)$ (Claim)

$={\displaystyle \bigcup _{i=1}^n}\widehat{\delta }(A_i,xa)$ (Definition 1.10(ii))

Therefore, the statement is also true for $\left|x\right|=k+1$.

Proposition 6.

$\widehat{\delta }(A,x)={\displaystyle \bigcup _{q\in A}}\widehat{\delta }(\left\{q\right\},x)$ for all $A\subset Q$.

$<Proof>$

LHS $=\widehat{\delta }\left({\displaystyle \bigcup _{q\in A}}\left\{q\right\},x\right)$

$={\displaystyle \bigcup _{q\in A}}\widehat{\delta }(\left\{q\right\},x)$ (Proposition 1.15)

= RHS

Proposition 7.

$\forall A,B,$ where $A\subset B\subset Q,\widehat{\delta }(A,x)\subset \widehat{\delta }(B,x)$

$<Proof>$

$B=A\cup (B\setminus A)$ (Set Theory)

$\widehat{\delta }(B,x)=\widehat{\delta }(A\cup (B\setminus A),x)$

$=\widehat{\delta }(A,x)\cup \widehat{\delta }(B\setminus A,x)$ (Proposition 1.15)

Therefore, $\widehat{\delta }(A,x)\subset \widehat{\delta }(B,x)$

Proposition 8.

For any two $NFAs$$N_1$ and $N_2$, where

$N_1=(Q_1,\Sigma ,{\delta }_1,q_1,F_1)$

$N_2=(Q_2,\Sigma ,{\delta }_2,q_2,F_2)$ and $Q_1\subset Q_2$

$\forall q\in Q_1,a\in {\Sigma }_{ϵ},{\delta }_1(q,a)\subset {\delta }_2(q,a)\Rightarrow \widehat{{\delta }_1}(\left\{q\right\},w)\subset \widehat{{\delta }_2}(\left\{q\right\},w)\forall w\in {\Sigma }_{ϵ}^{*}$

$<Proof>$

The proof is by induction on $\left|w\right|$.

For $\left|w\right|=0$, $w=ϵ$.

$\widehat{{\delta }_1}(\left\{q\right\},ϵ)=\left\{q\right\}$ and $\widehat{{\delta }_2}(\left\{q\right\},ϵ)=\left\{q\right\}$ (By Definition 1.10(i))

Therefore, $\widehat{{\delta }_1}(\left\{q\right\},ϵ)\subset \widehat{{\delta }_2}(\left\{q\right\},ϵ).$

The statement is true for $\left|w\right|=0$.

Assume the statement is true for $\left|w\right|=k\ge 0$.

That is, $\widehat{{\delta }_1}(\left\{q\right\},w)\subset \widehat{{\delta }_2}(\left\{q\right\},w)$ for $\left|w\right|=k\ge 0$.

For $k+1$,

$\widehat{{\delta }_1}(\left\{q\right\},wa)={\displaystyle \bigcup _{p\in \widehat{{\delta }_1}(\left\{q\right\},w)}}{\delta }_1(p,a)$

$\subset {\displaystyle \bigcup _{p\in \widehat{{\delta }_2}(\left\{q\right\},w)}}{\delta }_2(p,a)$(By Induction Hypothesis and ${\delta }_1(q,a)\subset {\delta }_2(q,a)$)

$=\widehat{{\delta }_2}(\left\{q\right\},wa)$(By Definition 1.10(ii))

Theorem 3.

(NFA acceptance)

$N=(Q,\Sigma ,\delta ,s_0,F)$ is an $NFA$.

$\forall w\in {\Sigma }_{ϵ}^{*}$ where $w=w_1{w}_2{w}_3\dots w_n;$ and

$(w_i\in {\Sigma }_{ϵ}$ for $1\le i\le n$ and $n\ge 1)$ or $(w=ϵ$ and $n=0).$

N accepts w if and only if $\widehat{\delta }(\left\{s_0\right\},w)\cap F\ne \varnothing$

In other words, N accepts w if and only if $(\exists r\in F$ s.t. $s_0\stackrel{w,\widehat{\delta }}{⟶}r)$

$<Proof>$

If N accepts w

$\exists r_0,r_1,r_2,\dots r_n\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n$ and $r_n\in F$.

$r_n\in \widehat{\delta }(\left\{s_0\right\},w)$(By Proposition 1.13)

Since $r_n$ is also in F,

$\widehat{\delta }(\left\{s_0\right\},w)\cap F\ne \varnothing$

Conversely, if $\widehat{\delta }(\left\{s_0\right\},w)\cap F\ne \varnothing$,

$\exists r_n\in \widehat{\delta }(\left\{s_0\right\},w)$ and $r_n\in F$.

$\exists r_0,r_1,r_2,\dots r_n\in Q$ s.t. $s_0=r_0\stackrel{w_1,\delta }{⟶}{r}_1\stackrel{w_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,\delta }{⟶}{r}_n$ ; $r_n\in F$ (Proposition 1.13)

Therefore, N accepts w.

3. Epsilon-Closure

The $ϵ$-Closure of a set of states is a collection of states that can be reached from a member of the given set of states via zero or a finite number of $ϵ$ transitions.

Formally, we define $ϵ$-Closure as follows.

Definition 9.

Let $N=(Q,\Sigma ,\delta ,s_0,F)$ be an $NFA$.

For any $R\subset Q,$ the ϵ-Closure of R is

$E\left(R\right)=\{q\in Q\mid p\stackrel{{ϵ}^i,\widehat{\delta }}{⟶}q$ for some $p\in R\}$ where

i is an integer $\ge 0$ and $p\stackrel{{ϵ}^0,\widehat{\delta }}{⟶}q$ means $p=q$.

Proposition 9.

$\forall A_i\subset Q,i=1,2,\dots n,n\in N,E\left({\displaystyle \bigcup _{i=1}^n}{A}_i\right)={\displaystyle \bigcup _{i=1}^n}E\left(A_i\right)$

$<Proof>$

Claim. $E(A_1\cup A_2)=E\left(A_1\right)\cup E\left(A_2\right)$

<Proof of Claim>

$q\in E(A_1\cup A_2)\iff \exists p\in A_1\cup A_2$ s.t. $p\stackrel{{ϵ}^i,\widehat{\delta }}{⟶}q$ where $i\ge 0$

$\iff ((\exists p\in A_1)\vee (\exists p\in A_2))\wedge (p\stackrel{{ϵ}^i,\widehat{\delta }}{⟶}q$ where $i\ge 0)$

$\iff (q\in E\left(A_1\right))\vee (q\in E\left(A_2\right))$

$\iff q\in E\left(A_1\right)\cup E\left(A_2\right)$

Therefore, $E(A_1\cup A_2)=E\left(A_1\right)\cup E\left(A_2\right)$

With this Claim and an induction argument, we can conclude Proposition 1.21.

4. The Equivalence of DFA and NFA

Lemma 1.

Let $N=(Q,\Sigma ,\delta ,q_0,F)$ be an $NFA$, $M=(Q^{\prime },\Sigma ,{\delta }^{\prime },q_0^{\prime },F^{\prime })$ be a $DFA$.

$Q^{\prime }=P\left(Q\right),q_0^{\prime }=E\left(\left\{q_0\right\}\right),F^{\prime }=\{R\in Q^{\prime }\mid R\cap F\ne \varnothing \}$

${\delta }^{\prime }:Q^{\prime }\times \Sigma ⟶Q^{\prime }$ such that

${\delta }^{\prime }(R,a)={\displaystyle \bigcup _{r\in R}}E\left(\delta (r,a)\right)\forall a\in \Sigma ,R\in Q^{\prime }$

Let $w=w_1{w}_2{w}_3\dots w_n$ such that if $n=0,w=ϵ$ and if $n\ge 1,$ then $w_i\ne ϵ\forall 1\le i\le n$.

Let $i_0,i_1,i_2,\dots i_n$ be integers $\ge 0,q_0,q_1,q_2,\dots q_n\in Q,p_1,p_2,p_3,\dots p_n\in Q$ and $q\in Q.$

The following holds:

$q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}{q}_1\stackrel{w_1,\delta }{⟶}{p}_1\stackrel{{ϵ}^{i_1},\widehat{\delta }}{⟶}{q}_2\stackrel{w_2,\delta }{⟶}{p}_2\stackrel{{ϵ}^{i_2},\widehat{\delta }}{⟶}{q}_3\dots q_{n-1}\stackrel{w_{n-1},\delta }{⟶}{p}_{n-1}\stackrel{{ϵ}^{i_{n-1}},\widehat{\delta }}{⟶}{q}_n\stackrel{w_n,\delta }{⟶}{p}_n\stackrel{{ϵ}^{i_n},\widehat{\delta }}{⟶}q.$

$⟺q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$

$<Proof>$

Proof is by induction on $\left|w\right|=n.$

Let $P\left(n\right)$ denote the statement of

$q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}{q}_1\stackrel{w_1,\delta }{⟶}{p}_1\stackrel{{ϵ}^{i_1},\widehat{\delta }}{⟶}{q}_2\stackrel{w_2,\delta }{⟶}{p}_2\stackrel{{ϵ}^{i_2},\widehat{\delta }}{⟶}{q}_3\dots q_{n-1}\stackrel{w_{n-1},\delta }{⟶}{p}_{n-1}\stackrel{{ϵ}^{i_{n-1}},\widehat{\delta }}{⟶}{q}_n\stackrel{w_n,\delta }{⟶}{p}_n\stackrel{{ϵ}^{i_n},\widehat{\delta }}{⟶}q.$

and $Q\left(n\right)$ denote the statement of $q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$ corresponding to $n\ge 0$.

For $\left|w\right|=n=0,w=ϵ$.

$P\left(0\right)⟺q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}q⟺q\in E\left(\left\{q_0\right\}\right)⟺q\in q_0^{\prime }(q_0^{\prime }=E\left(\left\{q_0\right\}\right))⟺q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },ϵ)$ (Definition 1.4(i))

$⟺q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)(w=ϵ)$

$⟺Q\left(0\right)$

Assume $P\left(k\right)\iff Q\left(k\right)$ for $k\ge 0$.

$P(k+1)$

$\iff q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}{q}_1\stackrel{w_1,\delta }{⟶}{p}_1\stackrel{{ϵ}^{i_1},\widehat{\delta }}{⟶}{q}_2\stackrel{w_2,\delta }{⟶}{p}_2\stackrel{{ϵ}^{i_2},\widehat{\delta }}{⟶}{q}_3\dots q_k\stackrel{w_k,\delta }{⟶}{p}_k\stackrel{{ϵ}^{i_k},\widehat{\delta }}{⟶}{q}_{k+1}\stackrel{w_{k+1},\delta }{⟶}{p}_{k+1}\stackrel{{ϵ}^{i_{k+1}},\widehat{\delta }}{⟶}q.$

$\iff P\left(k\right)$&$q_{k+1}\stackrel{w_{k+1},\delta }{⟶}{p}_{k+1}\stackrel{{ϵ}^{i_{k+1}},\widehat{\delta }}{⟶}q$

$\iff Q\left(k\right)$&$q_{k+1}\stackrel{w_{k+1},\delta }{⟶}{p}_{k+1}\stackrel{{ϵ}^{i_{k+1}},\widehat{\delta }}{⟶}q$ (Induction Hypothesis)

$\iff q_{k+1}\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$ where $\left|w\right|=k$&$q_{k+1}\stackrel{w_{k+1},\delta }{⟶}{p}_{k+1}\stackrel{{ϵ}^{i_{k+1}},\widehat{\delta }}{⟶}q$

$\iff q_{k+1}\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$ where $\left|w\right|=k$&$q\in E\left(\delta (q_{k+1},w_{k+1})\right)$

$\iff q\in {\displaystyle \bigcup _{r\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)}}E\left(\delta (r,w_{k+1})\right)$ where $\left|w\right|=k$

$\iff q\in {\delta }^{\prime }(\widehat{{\delta }^{\prime }}(q_0^{\prime },w),w_{k+1})$ where $\left|w\right|=k$

(Consider $R=\widehat{{\delta }^{\prime }}(q_0^{\prime },w),w_{k+1}=a$&${\delta }^{\prime }(R,a)\stackrel{def}{=}{\displaystyle \bigcup _{r\in R}}E\left(\delta (r,a)\right)$)

$\iff q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w{w}_{k+1})$ where $\left|w\right|=k$ (Definition 1.4(ii))

$\iff Q(k+1)$

This completes the proof of Lemma 1.22.

Theorem 4.

Every $NFA$ can be converted to an equivalent $DFA$.

$<Proof>$

Let $N=(Q,\Sigma ,\delta ,q_0,F)$ be an $NFA$.

Construct a DFA as follows.

$M=(Q^{\prime },\Sigma ,{\delta }^{\prime },q_0^{\prime },F^{\prime })$ where

$Q^{\prime }=P\left(Q\right),q_0^{\prime }=E\left(\left\{q_0\right\}\right),F^{\prime }=\{R\in Q^{\prime }\mid R\cap F\ne \varnothing \}$

${\delta }^{\prime }:Q^{\prime }\times \Sigma ⟶Q^{\prime }$ such that

${\delta }^{\prime }(R,a)={\displaystyle \bigcup _{r\in R}}E\left(\delta (r,a)\right)\forall a\in \Sigma ,R\in Q^{\prime }$

We claim that N and M are equivalent by showing that

$\forall w\in {\Sigma }_{ϵ}^{*},N$ accepts $w\iff M$ accepts w

The proof is divided into two cases, one with $w=ϵ$ and one with $w\ne ϵ$.

(i) $w=ϵ$

If N accepts w,

$\exists j\ge 0$ s.t. $q_0\stackrel{{ϵ}^j,\widehat{\delta }}{⟶}p$ and $p\in F$.

Therefore, $p\in E\left(\left\{q_0\right\}\right)$&$p\in F$.

Therefore, $p\in q_0^{\prime }$&$p\in F$.

Therefore, $q_0^{\prime }\cap F\ne \varnothing$.

Therefore, $q_0^{\prime }\in F^{\prime }$.

Therefore, the start state of M is also an accept state of M.

By definition, M accepts $ϵ(=w)$.

Conversely, if M accepts $w=ϵ$,

$q_0^{\prime }\in F^{\prime }$ (A $DFA$ accepts ϵ iff its start state is also an accept state.)

$q_0^{\prime }\cap F\ne \varnothing$ (By definition of $F^{\prime }$)

$\exists p\in q_0^{\prime }$ and $p\in F$.

Since $q_0^{\prime }=E\left(\left\{q_0\right\}\right),q_0\stackrel{{ϵ}^j,\widehat{\delta }}{⟶}p$ for some $j\ge 0$.

Since $p\in F$, N accepts ${ϵ}^j$, which is same as ϵ.

(ii) $w\ne ϵ$

$\exists w_i\ne ϵ,\forall 1\le i\le n,n\ge 1$ and

$w={ϵ}^{i_0}{w}_1{ϵ}^{i_1}{w}_2{ϵ}^{i_2}{w}_3{ϵ}^{i_3}\dots w_n{ϵ}^{i_n}$ for some integers $i_0,i_1,i_2,\dots i_n\ge 0$

If N accepts w,

$\exists q_0,q_1,q_2,\dots q_n\in Q,p_1,p_2,p_3,\dots p_n\in Q$ and $q\in Q.$ s.t.

$q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}{q}_1\stackrel{w_1,\delta }{⟶}{p}_1\stackrel{{ϵ}^{i_1},\widehat{\delta }}{⟶}{q}_2\stackrel{w_2,\delta }{⟶}{p}_2\stackrel{{ϵ}^{i_2},\widehat{\delta }}{⟶}{q}_3\dots q_{n-1}\stackrel{w_{n-1},\delta }{⟶}{p}_{n-1}\stackrel{{ϵ}^{i_{n-1}},\widehat{\delta }}{⟶}{q}_n\stackrel{w_n,\delta }{⟶}{p}_n\stackrel{{ϵ}^{i_n},\widehat{\delta }}{⟶}q$&$q\in F$.

By Lemma 1.22, $q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$ where $w=w_1{w}_2{w}_3\dots w_n$.

Therefore, $\widehat{{\delta }^{\prime }}(q_0^{\prime },w)\cap F\ne \varnothing$.

Therefore, $\widehat{{\delta }^{\prime }}(q_0^{\prime },w)\in F\prime$.

Therefore, M accepts $w$ ($DFA$ acceptance)

Conversely, if M accepts $w=w_1{w}_2{w}_3\dots w_n$,

$\widehat{{\delta }^{\prime }}(q_0^{\prime },w)\in F\prime$($DFA$ acceptance)

$\widehat{{\delta }^{\prime }}(q_0^{\prime },w)\cap F\ne \varnothing$ (Definition of $F\prime$)

$\exists q\in \widehat{{\delta }^{\prime }}(q_0^{\prime },w)$ and $q\in F$.

By Lemma 1.22,

$q_0\stackrel{{ϵ}^{i_0},\widehat{\delta }}{⟶}{q}_1\stackrel{w_1,\delta }{⟶}{p}_1\stackrel{{ϵ}^{i_1},\widehat{\delta }}{⟶}{q}_2\stackrel{w_2,\delta }{⟶}{p}_2\stackrel{{ϵ}^{i_2},\widehat{\delta }}{⟶}{q}_3\dots q_{n-1}\stackrel{w_{n-1},\delta }{⟶}{p}_{n-1}\stackrel{{ϵ}^{i_{n-1}},\widehat{\delta }}{⟶}{q}_n\stackrel{w_n,\delta }{⟶}{p}_n\stackrel{{ϵ}^{i_n},\widehat{\delta }}{⟶}q$&$q\in F$.

Therefore, N accepts $w={ϵ}^{i_0}{w}_1{ϵ}^{i_1}{w}_2{ϵ}^{i_2}{w}_3{ϵ}^{i_3}\dots w_n{ϵ}^{i_n}$

This completes the proof of Theorem 1.23.

Corollary 1.

A language is regular iff some $NFA$ recognizes it.

5. Regular Operators

Regular Languages are closed under the operation of Regular Operators.

Theorem 5.

L is regular $\Rightarrow {\Sigma }^{*}\setminus L$ is regular.

$<Proof>$

Let $M=(Q,\Sigma ,\delta ,q_0,F)$ be the $DFA$ that recognizes L.

That is, $L\left(M\right)=L$.

Define $M^{\prime }=(Q,\Sigma ,{\delta }^{\prime },q_0,Q\setminus F)$ where

${\delta }^{\prime }:Q\times \Sigma ⟶Q$ s.t. $\forall q\in Q,a\in \Sigma ,{\delta }^{\prime }(q,a)=\delta (q,a)$

$\forall w\in {\Sigma }^{*}\setminus L$,

$w\notin L⟹\widehat{\delta }(q_0,w)\notin F$

$⟹\widehat{\delta }(q_0,w)\in Q\setminus F$

$⟹\widehat{{\delta }^{\prime }}(q_0,w)\in Q\setminus F$ (Theorem 1.8)

$⟹M^{\prime }$ accepts w

Conversely, if $M^{\prime }$ accepts w, $\widehat{{\delta }^{\prime }}(q_0,w)\in Q\setminus F$

$\widehat{\delta }(q_0,w)\in Q\setminus F$ (Theorem 1.8)

Therefore, $\widehat{\delta }(q_0,w)\notin F$

Therefore, $w\notin L$ (because $w\in L⟹M$ accepts $w⟹\widehat{\delta }(q_0,w)\in F$)

$w\in {\Sigma }^{*}\setminus L$

Therefore, $w\in {\Sigma }^{*}\setminus L⟺M^{\prime }$ accepts w.

$L\left(M^{\prime }\right)={\Sigma }^{*}\setminus L$

${\Sigma }^{*}\setminus L$ is regular.

Theorem 6.

$L_1$ and $L_2$ are regular $⟹L_1\cap L_2$ is regular.

$<Proof>$

$\exists DFAs$$M_1$ and $M_2$ s.t. $L\left(M_1\right)=L_1$ and $L\left(M_2\right)=L_2$

Let $M_1=(Q_1,\Sigma ,{\delta }_1,s_0,F_1)$

$M_2=(Q_2,\Sigma ,{\delta }_2,s_0^{\prime },F_2)$

Define $M_3$ as follows.

$M_3=(Q_3,\Sigma ,{\delta }_3,s_0^{″},F_3)$

where $s_0^{″}=(s_0,s_0^{\prime }),Q_3=Q_1\times Q_2,F_3=F_1\times F_2$

${\delta }_3:Q_3\times \Sigma ⟶Q_3$ s.t.

${\delta }_3((q_1,q_2),a)=({\delta }_1(q_1,a),{\delta }_2(q_2,a))\forall q_1\in Q_1,q_2\in Q_2,a\in A$.

Claim. $\forall n\in N\cup \left\{0\right\},w\in {\Sigma }^{*}$, where $\left|w\right|=n$, if

(i) $s_0=r_0\stackrel{w_1,{\delta }_1}{⟶}{r}_1\stackrel{w_2,{\delta }_1}{⟶}{r}_2\dots r_{n-1}\stackrel{w_n,{\delta }_1}{⟶}{r}_n$

(ii) $s_0^{\prime }=r_0^{\prime }\stackrel{w_1,{\delta }_2}{⟶}{r}_1^{\prime }\stackrel{w_2,{\delta }_2}{⟶}{r}_2^{\prime }\dots r_{n-1}^{\prime }\stackrel{w_n,{\delta }_2}{⟶}{r}_n^{\prime }$

(iii) $s_0^{″}=r_0^{″}\stackrel{w_1,{\delta }_3}{⟶}{r}_1^{″}\stackrel{w_2,{\delta }_3}{⟶}{r}_2^{″}\dots r_{n-1}^{″}\stackrel{w_n,{\delta }_3}{⟶}{r}_n^{″}$

then $r_n^{″}=(r_n,r_n^{\prime })$.

Proof of Claim is by induction on n.

For $n=0$, (i), (ii) and (iii) become $s_0=r_0,s_0^{\prime }=r_0^{\prime }$, and $s_0^{″}=r_0^{″}$.

$s_0^{″}=(s_0,s_0^{\prime })$ (By definition of $M_3$.)

Therefore, $r_0^{″}=(r_0,r_0^{\prime })$

Assume the statement is true for $n=k\ge 0$.

(i), (ii) & (iii) for $n=k+1\Rightarrow$

$s_0=r_0\stackrel{w_1,{\delta }_1}{⟶}{r}_1\stackrel{w_2,{\delta }_1}{⟶}{r}_2\dots r_{k-1}\stackrel{w_k,{\delta }_1}{⟶}{r}_k\stackrel{w_{k+1},{\delta }_1}{⟶}{r}_{k+1}$

$s_0^{\prime }=r_0^{\prime }\stackrel{w_1,{\delta }_2}{⟶}{r}_1^{\prime }\stackrel{w_2,{\delta }_2}{⟶}{r}_2^{\prime }\dots r_{k-1}^{\prime }\stackrel{w_k,{\delta }_2}{⟶}{r}_k^{\prime }\stackrel{w_{k+1},{\delta }_2}{⟶}{r}_{k+1}^{\prime }$

$s_0^{″}=r_0^{″}\stackrel{w_1,{\delta }_3}{⟶}{r}_1^{″}\stackrel{w_2,{\delta }_3}{⟶}{r}_2^{″}\dots r_{k-1}^{″}\stackrel{w_k,{\delta }_3}{⟶}{r}_k^{″}\stackrel{w_{k+1},{\delta }_3}{⟶}{r}_{k+1}^{″}$

⇒ (i), (ii) & (iii) for $n=k$&$r_k\stackrel{w_{k+1},{\delta }_1}{⟶}{r}_{k+1}$&$r_k^{\prime }\stackrel{w_{k+1},{\delta }_2}{⟶}{r}_{k+1}^{\prime }$&$r_k^{″}\stackrel{w_{k+1},{\delta }_3}{⟶}{r}_{k+1}^{″}$

$\Rightarrow r_k^{″}=(r_k,r_k^{\prime })$&$r_{k+1}={\delta }_1(r_k,w_{k+1})$&$r_{k+1}^{\prime }={\delta }_2(r_k^{\prime },w_{k+1})$&$r_{k+1}^{″}={\delta }_3(r_k^{″},w_{k+1})$

(Induction Hypothesis)

$\Rightarrow r_{k+1}={\delta }_1(r_k,w_{k+1})$&$r_{k+1}^{\prime }={\delta }_2(r_k^{\prime },w_{k+1})$&$r_{k+1}^{″}={\delta }_3((r_k,r_k^{\prime }),w_{k+1})$

$\Rightarrow r_{k+1}={\delta }_1(r_k,w_{k+1})$&$r_{k+1}^{\prime }={\delta }_2(r_k^{\prime },w_{k+1})$&$r_{k+1}^{″}=({\delta }_1(r_k,w_{k+1}),{\delta }_2(r_k^{\prime },w_{k+1}))$

(Definition of ${\delta }_3)$

$\Rightarrow r_{k+1}^{″}=(r_{k+1},r_{k+1}^{\prime })$

We now need to show $L_1\cap L_2=L\left(M_3\right)$.

$\forall w\in L_1\cap L_2,w\in L_1$ and $w\in L_2$.

$w\in L_1\Rightarrow \exists s_0=r_0,r_1,r_2,\dots r_n$ s.t. $s_0=r_0\stackrel{w_1,{\delta }_1}{⟶}{r}_1\stackrel{w_2,{\delta }_1}{⟶}{r}_2\dots \dots r_{n-1}\stackrel{w_n,{\delta }_1}{⟶}{r}_n$&$r_n\in F_1$

$w\in L_2\Rightarrow \exists s_0^{\prime }=r_0^{\prime },r_1^{\prime },r_2^{\prime },\dots r_n^{\prime }$ s.t. $s_0^{\prime }=r_0^{\prime }\stackrel{w_1,{\delta }_2}{⟶}{r}_1^{\prime }\stackrel{w_2,{\delta }_2}{⟶}{r}_2^{\prime }\dots \dots r_{n-1}^{\prime }\stackrel{w_n,{\delta }_2}{⟶}{r}_n^{\prime }$&$r_n^{\prime }\in F_2$

Let $r_0^{″}=s_0^{″}=(s_0,s_0^{\prime })$

$r_1^{″}={\delta }_3(r_0^{″},w_1),\dots r_{i+1}^{″}={\delta }_3(r_i^{″},w_{i+1}),\dots r_n^{″}={\delta }_3(r_{n-1}^{″},w_n)$.

Therefore, $s_0^{″}=r_0^{″}\stackrel{w_1,{\delta }_3}{⟶}{r}_1^{″}\stackrel{w_2,{\delta }_3}{⟶}{r}_2^{″}\dots r_{n-1}^{″}\stackrel{w_n,{\delta }_3}{⟶}{r}_n^{″}$

By Claim, $r_n^{″}=(r_n,r_n^{\prime })$

Since $r_n\in F_1$ and $r_n^{\prime }\in F_2,r_n^{″}\in F_1\times F_2=F_3$.

Therefore, $M_3$ accepts w.

$w\in L\left(M_3\right)$

$L_1\cap L_2\subset L\left(M_3\right)$

Conversely, if $w\in L\left(M_3\right)$,

$\exists r_0^{″},r_1^{″},r_2^{″},\dots r_n^{″}\in Q_3$ s.t. $s_0^{″}=r_0^{″}\stackrel{w_1,{\delta }_3}{⟶}{r}_1^{″}\stackrel{w_2,{\delta }_3}{⟶}{r}_2^{″}\dots \dots r_{n-1}^{″}\stackrel{w_n,{\delta }_3}{⟶}{r}_n^{″}$&$r_n^{″}\in F_3$

Take

$r_0=s_0$;

$r_{i+1}={\delta }_1(r_i,w_{i+1})\forall i=0,1,2,\dots n-1$;

$r_0^{\prime }=s_0^{\prime }$;

$r_{i+1}^{\prime }={\delta }_2(r_i^{\prime },w_{i+1})\forall i=0,1,2,\dots n-1$.

Therefore,

$s_0=r_0\stackrel{w_1,{\delta }_1}{⟶}{r}_1\stackrel{w_2,{\delta }_1}{⟶}{r}_2\dots \dots r_{n-1}\stackrel{w_n,{\delta }_1}{⟶}{r}_n$

$s_0^{\prime }=r_0^{\prime }\stackrel{w_1,{\delta }_2}{⟶}{r}_1^{\prime }\stackrel{w_2,{\delta }_2}{⟶}{r}_2^{\prime }\dots \dots r_{n-1}^{\prime }\stackrel{w_n,{\delta }_2}{⟶}{r}_n^{\prime }$

By Claim, $r_n^{″}=(r_n,r_n^{\prime })$

Since $r_n^{″}\in F_3=F_1\times F_2,r_n\in F_1$ and $r_n^{\prime }\in F_2$.

$M_1$ accepts w and $M_2$ accepts w.

$w\in L\left(M_1\right)$ and $w\in L\left(M_2\right)$

$w\in L_1$ and $w\in L_2$

$w\in L_1\cap L_2$

$L\left(M_3\right)\subset L_1\cap L_2$

Combining both directions, $L\left(M_3\right)=L_1\cap L_2$

$L_1\cap L_2$ is regular.

Theorem 7.

$L_1$ and $L_2$ are regular $⟹L_1\cup L_2$ is regular.

$<Proof>$

From set theory, ${\Sigma }^{*}\setminus (L_1\cup L_2)=({\Sigma }^{*}\setminus L_1)\cap ({\Sigma }^{*}\setminus L_2)$

$L_1$ is regular $⟹{\Sigma }^{*}\setminus L_1$ is regular. (Theorem 1.25)

$L_2$ is regular $⟹{\Sigma }^{*}\setminus L_2$ is regular. (Theorem 1.25)

${\Sigma }^{*}\setminus L_1$ and ${\Sigma }^{*}\setminus L_2$ are regular $⟹({\Sigma }^{*}\setminus L_1)\cap ({\Sigma }^{*}\setminus L_2)$ is regular. (Theorem 1.26)

Therefore, ${\Sigma }^{*}\setminus (L_1\cup L_2)$ is regular.

Therefore, $L_1\cup L_2$ is regular. (Theorem 1.25)

Theorem 8.

Every $NFA$ can be converted to another $NFA$ with the following properties.

(i) There is only one accept state which has transition arrows coming in and no

transition arrows going out.

(ii) The accept state is different from the start state.

(iii) The start state has no arrows coming in from other states but only transition

arrows going out.

$<Proof>$

Let $N_1=(Q_1,\Sigma ,{\delta }_1,q_1,F_1)$ be the $NFA$ to be converted.

Define $NFA$, $N=(Q,\Sigma ,\delta ,q_0,\left\{q_a\right\})$ where $Q=Q_1\cup \{q_0,q_a\},q_0\ne q_a$ and

$\delta (q,x)=\left\{\begin{array}{ccc}\hfill \left\{q_1\right\}& if\hfill & (q,x,)=(q_0,ϵ)\hfill \\ \hfill \varnothing & if\hfill & q=q_{0}andx\ne ϵ\hfill \\ \hfill \varnothing & if\hfill & q=q_a\hfill \\ \hfill {\delta }_1(q,x)& if\hfill & q\in Q_1\setminus F_1\hfill \\ \hfill {\delta }_1(q,x)& if\hfill & q\in F_{1}andx\ne ϵ\hfill \\ \hfill {\delta }_1(q,x)\cup \left\{q_a\right\}& if\hfill & q\in F_{1}andx=ϵ\hfill \end{array}\right.$

It is clear that N satisfies conditions (i), (ii) and (iii).

Furthermore, ${\delta }_1(q,x)\subset \delta (q,x)\forall x\in {\Sigma }_{ϵ},q\in Q_1$ and hence

${\widehat{\delta }}_1(\left\{q\right\},w)\subset \widehat{\delta }(\left\{q\right\},w)\forall w\in {\Sigma }_{ϵ}^{*}$ by Proposition 1.18.

It remains to show that $\forall w\in {\Sigma }_{ϵ}^{*},N_1$ accepts $w\iff N$ accepts w.

For forward direction $"\Rightarrow "$,

Let $N_1$ accepts w.

$q_1\stackrel{w,{\widehat{\delta }}_1}{⟶}r$, $r\in F_1$.

Since ${\widehat{\delta }}_1(q_1,w)\subset \widehat{\delta }(q_1,w),q_1\stackrel{w,\widehat{\delta }}{⟶}r,r\in F_1$.

Since $\delta (q_0,ϵ)=\left\{q_1\right\},q_0\stackrel{ϵ,\delta }{⟶}{q}_1$.

Furthermore, since $\delta (q,ϵ)={\delta }_1(q,ϵ)\cup \left\{q_a\right\}\forall q\in F_1$, $\delta (r,ϵ)={\delta }_1(r,ϵ)\cup \left\{q_a\right\}$.

Therefore, $q_a\in \delta (r,ϵ)$.

That is, $r\stackrel{ϵ,\delta }{⟶}{q}_a$

Therefore, $q_0\stackrel{ϵ,\delta }{⟶}{q}_1\stackrel{w,\widehat{\delta }}{⟶}r\stackrel{ϵ,\delta }{⟶}{q}_a$.

Therefore N accepts $ϵwϵ$ which is the same as w.

Therefore, $N_1$ accepts $w\Rightarrow N$ accepts w.

Conversely, if N accepts $w=x_1{x}_2\dots x_n$ where $x_i\in {\Sigma }_{ϵ}$ for $n\ge 1$&$1\le i\le n$.

(Note that $w=ϵ$ if $x_i=ϵ\forall i$.)

$\exists r_0,r_1,r_2,\dots r_n\in Q$ s.t.

$q_0=r_0\stackrel{x_1,\delta }{⟶}{r}_1\stackrel{x_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{x_n,\delta }{⟶}{r}_n$&$r_n\in \left\{q_a\right\}$

Since the only way to transition to $q_a$ using δ is from a state in $F_1$ via the ϵ arrow, we must have $r_{n-1}\in F_1$&$x_n=ϵ$.

Since the only way to transition out of $q_0(=r_0)$ using δ is via an ϵ arrow, we must have $x_1=ϵ$.

Since $\delta (q_0,ϵ)=\left\{q_1\right\}$, we must have $r_1=q_1$.

We now can rewrite the above computation as

$q_0=r_0\stackrel{ϵ,\delta }{⟶}{q}_1\stackrel{x_2,\delta }{⟶}{r}_2\dots r_{n-2}\stackrel{x_{n-1},\delta }{⟶}{r}_{n-1}\stackrel{ϵ,\delta }{⟶}{q}_a$&$r_{n-1}\in F_1$.

For all $1\le j\le n-2,r_j\notin \{q_0,q_a\}$ because $r_j$ has both incoming and outgoing arrows.

Therefore, $r_j\in Q_1$.

Claim. $r_j\stackrel{x_{j+1},{\delta }_1}{⟶}{r}_{j+1}\forall 1\le j\le n-2$.

Since $r_j\in Q_1,\delta (r_j,x_{j+1})={\delta }_1(r_j,x_{j+1})$ or ${\delta }_1(r_j,x_{j+1})\cup \left\{q_a\right\}$ by definition of δ.

$r_j\stackrel{x_{j+1},\delta }{⟶}{r}_{j+1}$

$\Rightarrow r_{j+1}\in \delta (r_j,x_{j+1})$

$\Rightarrow r_{j+1}\in {\delta }_1(r_j,x_{j+1})$ or $r_{j+1}\in {\delta }_1(r_j,x_{j+1})\cup \left\{q_a\right\}$

$\Rightarrow r_{j+1}\in {\delta }_1(r_j,x_{j+1})$ or $r_{j+1}\in {\delta }_1(r_j,x_{j+1})$ (because $r_{j+1}\ne q_a$)

$\Rightarrow r_{j+1}\in {\delta }_1(r_j,x_{j+1})$

$\Rightarrow r_j\stackrel{x_{j+1},{\delta }_1}{⟶}{r}_{j+1}$

The computation now becomes

$q_0=r_0\stackrel{ϵ,\delta }{⟶}{q}_1\stackrel{x_2,{\delta }_1}{⟶}{r}_2\dots r_{n-2}\stackrel{x_{n-1},{\delta }_1}{⟶}{r}_{n-1}\stackrel{ϵ,\delta }{⟶}{q}_a$&$r_{n-1}\in F_1$.

Therefore, $q_1\stackrel{x_2,{\delta }_1}{⟶}{r}_2\dots r_{n-2}\stackrel{x_{n-1},{\delta }_1}{⟶}{r}_{n-1}$&$r_{n-1}\in F_1$.

Therefore, $N_1$ accepts $x_2{x}_3\dots \dots x_{n-1}$.

Therefore, $N_1$ accepts $w=x_1{x}_2{x}_3\dots \dots x_{n-1}{x}_n$ because $x_1=x_n=ϵ$.

Therefore, N accepts $w\Rightarrow N_1$ accepts w.

This completes the proof of Theorem 1.28.

Theorem 9.

For any regular languages $L_1$ and $L_2$, the language $L_1{L}_2$ is regular.

$<Proof>$

Since $L_1$ and $L_2$ are regular, there exist $NFAs$$N_1,N_2$ that recognize $L_1$ and $L_2$.

By Theorem 1.28, we can start with $N_1$ and $N_2$ defined as follows.

$N_1=(Q_1,\Sigma ,{\delta }_1,q_{1s},\left\{q_{1a}\right\})$ where

$q_{1s}\ne q_{1a},q_{1s}\notin {\delta }_1(q,x)\forall q\in Q_1,x\in {\Sigma }_{ϵ}$ and ${\delta }_1(q_{1a},x)=\varnothing \forall x\in {\Sigma }_{ϵ}$.

$N_2=(Q_2,\Sigma ,{\delta }_2,q_{2s},\left\{q_{2a}\right\})$ where

$q_{2s}\ne q_{2a},q_{2s}\notin {\delta }_2(q,x)\forall q\in Q_2,x\in {\Sigma }_{ϵ}$ and ${\delta }_2(q_{2a},x)=\varnothing \forall x\in {\Sigma }_{ϵ}$.

We can further assume that $Q_1\cap Q_2=\varnothing$ because we can always replace $Q_1$ with a set of objects which are completely different from those in $Q_2$ without affecting the function of $N_1$.

Now construct $N=(Q,\Sigma ,\delta ,q_{1s},\left\{q_{2a}\right\})$ where $Q=Q_1\cup Q_2$.

$\delta (q,x)=\left\{\begin{array}{ccc}\hfill {\delta }_1(q,x)& if\hfill & q\in Q_1\setminus \left\{q_{1a}\right\}\hfill \\ \hfill {\delta }_1(q_{1a},x)& if\hfill & q=q_{1a}\&x\ne ϵ\hfill \\ \hfill {\delta }_1(q_{1a},x)\cup \left\{q_{2s}\right\}& if\hfill & q=q_{1a}\&x=ϵ\hfill \\ \hfill {\delta }_2(q,x)& if\hfill & q\in Q_2\hfill \end{array}\right.$

We now need to show $L\left(N\right)=L_1{L}_2$.

If $w\in L_1{L}_2,w=w_1{w}_2$ where $w_1,w_2\in {\Sigma }_{ϵ}^{*}$ and $w_1\in L_1,w_2\in L_2$.

Since $N_1$ recognizes $L_1$ and $N_2$ recognizes $L_2,N_1$ accepts $w_1$ and $N_2$ accepts $w_2$.

$\exists r_1\in \left\{q_{1a}\right\}$ and $r_2\in \left\{q_{2a}\right\}$ such that

$q_{1s}\stackrel{w_1,{\widehat{\delta }}_1}{⟶}{r}_1$ and $q_{2s}\stackrel{w_2,{\widehat{\delta }}_2}{⟶}{r}_2$ (By Theorem 1.19 of $NFA$ Acceptance)

$q_{1s}\stackrel{w_1,\widehat{\delta }}{⟶}{q}_{1a}$ and $q_{2s}\stackrel{w_2,\widehat{\delta }}{⟶}{q}_{2a}$ (Proposition 1.18 and $r_1=q_{1a};r_2=q_{2a}$).

By definition of $\delta ,q_{2s}\in \delta (q_{1a},ϵ)$.

Therefore,

$q_{1s}\stackrel{w_1,\widehat{\delta }}{⟶}{q}_{1a}\stackrel{ϵ,\delta }{⟶}{q}_{2s}\stackrel{w_2,\widehat{\delta }}{⟶}{q}_{2a}$.

Therefore, N accepts $w_1ϵw_2,$ which is the same as $w_1{w}_2$.

$L_1{L}_2\subset L\left(N\right)$

Conversely, if N accepts $w=x_1{x}_2\dots x_n,$ where $x_1,x_2,\dots x_n\in {\Sigma }_{ϵ}$ for $n\ge 1$,

$\exists r_0,r_1,r_2,\dots r_n\in Q$ such that

$q_{1s}=r_0\stackrel{x_1,\delta }{⟶}{r}_1\stackrel{x_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{x_n,\delta }{⟶}{r}_n$&$r_n=q_{2a}$.

(Note that $w=ϵ$ if $x_i=ϵ\forall i$).

Since the only way to transition from a state of $N_1$ to a state of $N_2$ is via $q_{1a}$ to $q_{2s}$ using the ϵ arrow, ∃ an $r_i=q_{1a}$ and $r_{i+1}=q_{2s}$ such that $x_{i+1}=ϵ$ and the computation becomes

$q_{1s}=r_0\stackrel{x_1,\delta }{⟶}{r}_1\stackrel{x_2,\delta }{⟶}{r}_2\dots r_{i-1}\stackrel{x_i,\delta }{⟶}{q}_{1a}\stackrel{ϵ,\delta }{⟶}{q}_{2s}\stackrel{x_{i+2},\delta }{⟶}{r}_{i+2}\dots r_{n-1}\stackrel{x_n,\delta }{⟶}{r}_n;r_n=q_{2a}$.

Claim 1. $r_0,r_1,r_2,\dots r_{i-1}\in Q_1$.

<Proof of Claim 1>

$q_{1s}=r_0\Rightarrow r_0\in Q_1$.

Assume for contradiction that $r_{i-1}\notin Q_1$.

Then $r_{i-1}\in Q_2$.

$r_{i-1}\stackrel{x_i,\delta }{⟶}{q}_{1a}$

$\Rightarrow r_{i-1}\stackrel{x_i,{\delta }_2}{⟶}{q}_{1a}$ ($\delta (q,x)={\delta }_2(q,x)$ if $q\in Q_2$)

$\Rightarrow q_{1a}\in Q_2$

⇒ Contradiction

Therefore, $r_{i-1}\in Q_1$.

With similar and inductive argument, we can conclude $r_{i-2},\dots r_2,r_1$ are all in $Q_1$.

Claim 2. $r_j\ne q_{1a}\forall 0\le j\le i-1$.

Assume for contradiction $r_j=q_{1a}$ for some $0\le j\le i-1$.

Therefore, $r_j\stackrel{x_{j+1},\delta }{⟶}{r}_{j+1}\iff q_{1a}\stackrel{x_{j+1},\delta }{⟶}{r}_{j+1}\iff r_{j+1}\in \delta (q_{1a},x_{j+1})$.

By definition of $\delta ,\delta (q_{1a},x_{j+1})$

$={\delta }_1(q_{1a},x_{j+1})$ or ${\delta }_1(q_{1a},x_{j+1})\cup \left\{q_{2s}\right\}$

$=\varnothing$ or $\varnothing \cup \left\{q_{2s}\right\}$

$=\varnothing$ or $\left\{q_{2s}\right\}$

Therefore, $r_{j+1}\in \varnothing$ or $r_{j+1}\in \left\{q_{2s}\right\}$.

Either of these leads to a contradiction.

Therefore, $r_j\ne q_{1a}\forall 0\le j\le i-1$.

Combining Claim 1 and Claim 2, $r_j\in Q_1\setminus \left\{q_{1a}\right\}\forall 0\le j\le i-1$.

By definition of $\delta ,\delta (r_j,x)={\delta }_1(r_j,x)\forall 0\le j\le i-1$.

Therefore, computation $q_{1s}=r_0\stackrel{x_1,\delta }{⟶}{r}_1\stackrel{x_2,\delta }{⟶}{r}_2\dots r_{i-1}\stackrel{x_i,\delta }{⟶}{q}_{1a}$ can be replaced by computation

$q_{1s}=r_0\stackrel{x_1,{\delta }_1}{⟶}{r}_1\stackrel{x_2,{\delta }_1}{⟶}{r}_2\dots r_{i-1}\stackrel{x_i,{\delta }_1}{⟶}{q}_{1a}$.

Therefore, $N_1$ accepts $w_1=x_1{x}_2\dots x_i$.

$w_1\in L\left(N_1\right)=L_1$.

Claim 3. $r_j\in Q_2\forall i+2\le j\le n-1$.

<Proof of Claim 3>

$q_{2s}\stackrel{x_{i+2},\delta }{⟶}{r}_{i+2}$

$\Rightarrow r_{i+2}\in \delta (q_{2s},x_{i+2})$

$\Rightarrow r_{i+2}\in {\delta }_2(q_{2s},x_{i+2})(\delta (q,x)={\delta }_2(q,x)$ if $q\in Q_2)$

$\Rightarrow q_{2s}\stackrel{x_{i+2},{\delta }_2}{⟶}{r}_{i+2}$

$\Rightarrow r_{i+2}\in Q_2$.

With similar and inductive argument, we can show that $r_{i+3},\dots r_{n-1}$ are all in $Q_2$.

Therefore, computation $q_{2s}\stackrel{x_{i+2},\delta }{⟶}{r}_{i+2}\dots r_{n-1}\stackrel{x_n,\delta }{⟶}{r}_n;r_n=q_{2a}$ can be replaced by computation

$q_{2s}\stackrel{x_{i+2},{\delta }_2}{⟶}{r}_{i+2}\dots r_{n-1}\stackrel{x_n,{\delta }_2}{⟶}{r}_n;r_n=q_{2a}$

Therefore, $N_2$ accepts $w_2=x_{i+2}{x}_{i+3}\dots x_n$.

$w_2\in L\left(N_2\right)=L_2$.

$w_1{w}_2\in L_1{L}_2$.

$w=x_1{x}_2\dots x_i{x}_{i+1}{x}_{i+2}\dots x_n$

$=w_1{x}_{i+1}{w}_2$

$=w_1{w}_2(x_{i+1}=ϵ)$

Therefore, $w\in L_1{L}_2$.

Therefore, $L\left(N\right)\subset L_1{L}_2$.

Combining both directions, $L\left(N\right)=L_1{L}_2$.

Theorem 10.

For any regular language $L,L^{*}$ is regular.

$<Proof>$

Let $N_1$ be the $NFA$ that recognizes L.

By Theorem 1.28, we can start with an $N_1$ defined as follows.

$N_1=(Q_1,\Sigma ,T_1,q_1,\left\{q_a\right\})$ where

$q_1\ne q_a,q_1\notin T_1(q,x)\forall q\in Q_1,x\in {\Sigma }_{ϵ}$ and $T_1(q_a,x)=\varnothing \forall x\in {\Sigma }_{ϵ}$.

Let $N=(Q,\Sigma ,T,q_0,\{q_a,q_0\})$ such that $Q=Q_1\cup \left\{q_0\right\}$.

$T(q,x)=\left\{\begin{array}{ccc}\hfill T_1(q,x)& if\hfill & q\in Q_1\setminus \left\{q_a\right\}\hfill \\ \hfill \left\{q_1\right\}\cup T_1(q_a,ϵ)& if\hfill & q=q_a\&x=ϵ\hfill \\ \hfill T_1(q_a,x)& if\hfill & q=q_a\&x\ne ϵ\hfill \\ \hfill \left\{q_1\right\}& if\hfill & q=q_0\&x=ϵ\hfill \\ \hfill \varnothing & if\hfill & q=q_0\&x\ne ϵ\hfill \end{array}\right.$

We need to show $w\in L^{*}\iff N$ accepts w.

If $w\in L^{*}$,

$w\in L^M$ for some $M\ge 0$.

If $M=0,w\in L^0=\left\{ϵ\right\}$.

Therefore, $w=ϵ$.

ϵ is accepted by N because N has a start state that is also an accept state.

For $M\ge 1$,

let $w=w_1{w}_2\dots w_M$ with each $w_i\in L$ for $1\le i\le M$.

Therefore, $N_1$ accepts $w_i$ for each i.

For each $i,q_1\stackrel{w_i,\widehat{T_1}}{⟶}{q}_a$ (By Theorem 1.19 of $NFA$ Acceptance)

For each $i,q_1\stackrel{w_i,\widehat{T}}{⟶}{q}_a$ (Proposition 1.18)

Since $T(q_a,ϵ)=\left\{q_1\right\}\cup T_1(q_a,ϵ)$,

$q_1\in \left\{q_1\right\}\cup T_1(q_a,ϵ)\Rightarrow q_1\in T(q_a,ϵ)\Rightarrow q_a\stackrel{ϵ,T}{⟶}{q}_1$.

Therefore,

$q_0\stackrel{ϵ,T}{⟶}{q}_1\stackrel{w_1,\widehat{T}}{⟶}{q}_a\stackrel{ϵ,T}{⟶}{q}_1\stackrel{w_2,\widehat{T}}{⟶}{q}_a\stackrel{ϵ,T}{⟶}{q}_1\dots q_a\stackrel{ϵ,T}{⟶}{q}_1\stackrel{w_M,\widehat{T}}{⟶}{q}_a$

Therefore, N accepts $ϵw_1ϵw_2\dots ϵw_M=w_1{w}_2\dots w_M=w$.

Therefore, $w\in L^{*}\Rightarrow N$ accepts w.

Conversely, if N accepts $w=x_1{x}_2{x}_3\dots x_n$ where $x_i\in {\Sigma }_{ϵ}$ for $1\le i\le n\&n\ge 1$.

(Note that $w=ϵ$ if $x_i=ϵ\forall i$.)

$\exists r_0,r_1,r_2,\dots r_n\in Q$ such that

$q_0=r_0\stackrel{x_1,T}{⟶}{r}_1\stackrel{x_2,T}{⟶}{r}_2\dots r_{n-1}\stackrel{x_n,T}{⟶}{r}_n$&$r_n\in \{q_0,q_a\}$.

Since $T(q_0,x)=\varnothing$ if $x\ne ϵ,x_1=ϵ$.

Furthermore, $T(q_0,ϵ)=\left\{q_1\right\}$.

Therefore, $r_1=q_1$.

$r_n\in \{q_0,q_a\}\Rightarrow r_n=q_a$ because $q_0$ has no incoming arrows.

The computation now becomes

$q_0\stackrel{ϵ,T}{⟶}{q}_1\stackrel{x_2,T}{⟶}{r}_2\dots r_{n-1}\stackrel{x_n,T}{⟶}{q}_a$.

Therefore, $q_1\stackrel{x_2,T}{⟶}{r}_2\dots r_{n-1}\stackrel{x_n,T}{⟶}{q}_a$.

Claim 1:

For the computation, $q_0=r_0\stackrel{x_1,T}{⟶}{r}_1\stackrel{x_2,T}{⟶}{r}_2\dots r_i\stackrel{x_{i+1},T}{⟶}{r}_{i+1}\dots r_{n-1}\stackrel{x_n,T}{⟶}{r}_n$&$r_n=q_a$,

if $\exists r_i=q_a$ for $1<i<n-1$, then $r_{i+1}=q_1$&$x_{i+1}=ϵ$.

<Proof of Claim 1>

$r_i\stackrel{x_{i+1},T}{⟶}{r}_{i+1}\Rightarrow q_a\stackrel{x_{i+1},T}{⟶}{r}_{i+1}\Rightarrow r_{i+1}\in T(q_a,x_{i+1})$.

$T(q_a,x_{i+1})$

$=T_1(q_a,x_{i+1})$ or $T_1(q_a,x_{i+1})\cup \left\{q_1\right\}$ (by definition of T)

$=\varnothing$ or $\varnothing \cup \left\{q_1\right\}$ (by definition of $N_1$)

$=\varnothing$ or $\left\{q_1\right\}$

Therefore, $r_{i+1}\in \varnothing$ or $r_{i+1}\in \left\{q_1\right\}$.

Therefore, $r_{i+1}\in \left\{q_1\right\}$ and hence $r_{i+1}=q_1$.

Therefore, $r_i\stackrel{x_{i+1},T}{⟶}{r}_{i+1}$

$\Rightarrow q_a\stackrel{x_{i+1},T}{⟶}{q}_1$

$\Rightarrow q_1\in T_1(q_a,x_{i+1})$ if $x_{i+1}\ne ϵ$

$\Rightarrow q_1\in \varnothing$ if $x_{i+1}\ne ϵ$ (by definition of $N_1$)

⇒ Contradiction if $x_{i+1}\ne ϵ$.

Therefore, $x_{i+1}=ϵ$.

Claim 2:

For any computation $q_1\stackrel{x_1,T}{⟶}{s}_1\stackrel{x_2,T}{⟶}{s}_2\dots s_i\stackrel{x_{i+1},T}{⟶}{s}_{i+1}\dots s_{n-1}\stackrel{x_n,T}{⟶}{s}_n\stackrel{x_{n+1},T}{⟶}{q}_a$,

if ∃ no $q_a$ in between $q_1$ and $q_a$, that is $s_i\ne q_a$ for $1\le i\le n$, then

$q_1\stackrel{w,\widehat{T_1}}{⟶}{q}_a$ for some $w\in {\Sigma }_{ϵ}^{*}$.

<Proof of Claim 2>

$q_0\notin \{s_1,s_2,\dots s_n\}$ because $q_0$ has no incoming arrows.

Therefore, $s_1,s_2,\dots s_n\in Q_1$.

Therefore, $s_1,s_2,\dots s_n\in Q_1\setminus \left\{q_a\right\}$.

By definition of $T,T(q,x)=T_1(q,x)$ if $q\in Q_1\setminus \left\{q_a\right\}$.

Therefore, $T(s_i,x)=T_1(s_i,x)$ for $1\le i\le n$&$x\in {\Sigma }_{ϵ}$.

The given computation can be replaced by

$q_1\stackrel{x_1,T_1}{⟶}{s}_1\stackrel{x_2,T_1}{⟶}{s}_2\dots s_i\stackrel{x_{i+1},T_1}{⟶}{s}_{i+1}\dots s_{n-1}\stackrel{x_n,T_1}{⟶}{s}_n\stackrel{x_{n+1},T_1}{⟶}{q}_a$,

$q_1\stackrel{w,\widehat{T_1}}{⟶}{q}_a$ where $w=x_1{x}_2{x}_3\dots x_{n+1}$.

Back to computation $q_1\stackrel{x_2,T}{⟶}{r}_2\stackrel{x_3,T}{⟶}{r}_3\dots r_{n-1}\stackrel{x_n,T}{⟶}{q}_a$.

Let m be the number of $q_a$’s in between $q_1$&$q_a$.

If $m=0$, by Claim 2, $q_1\stackrel{w^{\prime },\widehat{T_1}}{⟶}{q}_a$ where $w^{\prime }=x_2{x}_3\dots x_n$.

$N_1$ accepts $w^{\prime }$.

$w=x_1{w}^{\prime }=ϵw^{\prime }=w^{\prime }$.

$N_1$ accepts w.

$w\in L\subset L^{*}$.

For $m\ge 1,\exists r_{j_1}=r_{j_2}=\dots r_{j_m}=q_a$.

By Claim 1, $r_{j_1+1}=r_{j_2+1}=\dots r_{j_m+1}=q_1$.

$q_1\stackrel{w_1,\widehat{T_1}}{⟶}{r}_{j_1}=q_a$ (Claim 2)

$q_a=r_{j_1}\stackrel{ϵ,T}{⟶}{r}_{j_1+1}=q_1$ (Claim 1)

$q_1=r_{j_1+1}\stackrel{w_2,\widehat{T_1}}{⟶}{r}_{j_2}=q_a$ (Claim 2)

$q_a=r_{j_2}\stackrel{ϵ,T}{⟶}{r}_{j_2+1}=q_1$ (Claim 1)

   ⋮

   ⋮

$q_1=r_{j_{m-1}+1}\stackrel{w_m,\widehat{T_1}}{⟶}{r}_{j_m}=q_a$ (Claim 2)

$q_a=r_{j_m}\stackrel{ϵ,T}{⟶}{q}_1\stackrel{w_{m+1},\widehat{T_1}}{⟶}{q}_a$ (Claim 1 & Claim 2)

Therefore, $N_1$ accepts $w_1,w_2,\dots w_m,w_{m+1}$.

$w_1,w_2,\dots w_m,w_{m+1}\in L$.

$w_1{w}_2\dots w_m{w}_{m+1}\in L^{m+1}$

However, $x_2{x}_3\dots x_n=w_1ϵw_2ϵ\dots ϵw_mϵw_{m+1}=w_1{w}_2\dots w_m{w}_{m+1}$.

$w=x_1{x}_2{x}_3\dots x_n=ϵx_2{x}_3\dots x_n=x_2{x}_3\dots x_n=w_1{w}_2\dots w_m{w}_{m+1}$.

Therefore, $w\in L^{m+1}\subset L^{*}$.

Therefore, N accepts $w\Rightarrow w\in L^{*}$.

Combining both directions, $w\in L^{*}\iff N$ accepts w.

This completes the proof of Theorem 1.30.

Definition 10.

For any string $w=x_1{x}_2\dots x_n$, where $x_i\in {\Sigma }_{ϵ}$ for each i, the reverse of w, written $w^R$ is the string $x_n{x}_{n-1}\dots x_1$.

For any language A, $A^R\stackrel{def}{=}\{w^R\mid w\in A\}$.

Theorem 11.

For any language A, A is regular iff $A^R$ is regular.

$<Proof>$

Since A is regular, there is an $NFA$, $N_A$ that recognizes it.

Let $N_A=(Q_A,\Sigma ,{\delta }_A,q_A,F_A)$.

Construct $N_{A^R}=(Q_A\cup \left\{q_s\right\},\Sigma ,{\delta }_{A^R},q_s,\left\{q_A\right\})$ where $q_s\notin Q_A$ such that

${\delta }_{A^R}(q,x)=\left\{\begin{array}{ccc}\hfill F_A& if\hfill & (q,x)=(q_s,ϵ)\hfill \\ \hfill \varnothing & if\hfill & q=q_s\&x\ne ϵ\hfill \\ \hfill \{p\in Q_A\mid q\in {\delta }_A(p,x)\}& if\hfill & q\in Q_A\hfill \end{array}\right.$

From the third row of this definition, it immediately follows that

$p\in {\delta }_{A^R}(q,x)\iff q\in {\delta }_A(p,x)$ or

$q\stackrel{x,{\delta }_{A^R}}{⟶}p\iff p\stackrel{x,{\delta }_A}{⟶}q\dots \dots (*)$.

Claim: ∃ a computation path for w from p to q via transition function ${\delta }_A$ iff ∃ a computation path for $w^R$ from q to p via transition function ${\delta }_{A^R}$.

That is, $p\stackrel{w,\widehat{{\delta }_A}}{⟶}q\iff q\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}p$.

This Claim can be proved by induction on $\left|w\right|$.

For $\left|w\right|=1$, $w=w^R=x$ where $x\in {\Sigma }_{ϵ}$.

From $(*)$, $q\stackrel{x,{\delta }_{A^R}}{⟶}p\iff p\stackrel{x,{\delta }_A}{⟶}q$

Therefore, the statement is true for $\left|w\right|=1$.

Assume the statement is true for $\left|w\right|=k$ where $k\ge 1$.

That is , $p\stackrel{w,\widehat{{\delta }_A}}{⟶}q\iff q\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}p$ for $\left|w\right|=k$.

$p\stackrel{wx,\widehat{{\delta }_A}}{⟶}q$

$\iff p\stackrel{w,\widehat{{\delta }_A}}{⟶}{q}^{\prime }\stackrel{x,{\delta }_A}{⟶}q$ (Proposition 1.12)

$\iff p\stackrel{w,\widehat{{\delta }_A}}{⟶}{q}^{\prime }$ and $q^{\prime }\stackrel{x,{\delta }_A}{⟶}q$

$\iff q^{\prime }\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}p$ and $q\stackrel{x,{\delta }_{A^R}}{⟶}{q}^{\prime }$ (Induction Hypothesis and (*))

$\iff q\stackrel{x,{\delta }_{A^R}}{⟶}{q}^{\prime }\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}p$

$\iff q\stackrel{x{w}^R,\widehat{{\delta }_{A^R}}}{⟶}p$ (Proposition 1.12)

$\iff q\stackrel{{\left(wx\right)}^R,\widehat{{\delta }_{A^R}}}{⟶}p$ ($x{w}^R={\left(wx\right)}^R$)

The statement is true for $\left|w\right|=k+1$ and the proof of Claim is complete.

To prove that $A^R$ is regular, we need to prove that

$w^R\in A^R$ iff $N_{A^R}$ accepts $w^R$.

If $w^R\in A^R$, $w\in A$.

Since $N_A$ accepts w, $q_A\stackrel{w,\widehat{{\delta }_A}}{⟶}q$, $q\in F_A$ (Theorem 1.19 – $NFA$ acceptance)

By Claim, $q\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}{q}_A$

Since ${\delta }_{A^R}(q_s,ϵ)=F_A$, and $q\in F_A$,

$q_s\stackrel{ϵ,{\delta }_{A^R}}{⟶}q$.

Therefore, $q_s\stackrel{ϵ,{\delta }_{A^R}}{⟶}q\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}{q}_A$.

$N_{A^R}$ accepts $ϵw^R$ (Theorem 1.19 – $NFA$ acceptance)

$N_{A^R}$ accepts $w^R$.

Conversely, if $N_{A^R}$ accepts $w^R$,

$N_{A^R}$ accepts $ϵw^R$.

$q_s\stackrel{ϵw^R,\widehat{{\delta }_{A^R}}}{⟶}{q}_A$(Theorem 1.19 – $NFA$ acceptance)

$q_s\stackrel{ϵ,{\delta }_{A^R}}{⟶}q\stackrel{w^R,\widehat{{\delta }_{A^R}}}{⟶}{q}_A$ (Proposition 1.12)

Since ${\delta }_{A^R}(q_s,ϵ)=F_A$, $q\in F_A$.

$q_A\stackrel{w,\widehat{{\delta }_A}}{⟶}q$, and $q\in F_A$ (Claim)

Therefore, $N_A$ accepts $w$(Theorem 1.19 – $NFA$ acceptance)

$w\in A$($N_A$ recognizes A)

$w^R\in A^R$.

$w^R\in A^R$ iff $N_{A^R}$ accepts $w^R$.

We have proved that A is regular $\Rightarrow A^R$ is regular.

On the other hand, sine ${\left(A^R\right)}^R=A$,

$A^R$ is regular $\Rightarrow {\left(A^R\right)}^R$ is regular $\Rightarrow A$ is regular.

Therefore, A is regular iff $A^R$ is regular.

6. Regular Expression

Definition 11.

(Regular Expression)

Let Σ be a finite alphabet.

${\Re }_{\Sigma }$ is a set with the following properties:

(a) $R\in {\Re }_{\Sigma }$ iff R is one of the following:

  (i) a for some $a\in \Sigma$

  (ii) $\widehat{ϵ}$

  (iii) $\widehat{\varnothing }$

  (iv) $R_1\widehat{\cup }{R}_2$ for some $R_1,R_2\in {\Re }_{\Sigma }$

  (v) $R_1\widehat{•}{R}_2$ for some $R_1,R_2\in {\Re }_{\Sigma }$

  (vi) $R_1^{\widehat{*}}$ for some $R_1\in {\Re }_{\Sigma }$

  where $\widehat{\cup },\widehat{•}$ and $\widehat{*}$ are operations in ${\Re }_{\Sigma }$ with

   $\widehat{\cup }:{\Re }_{\Sigma }\times {\Re }_{\Sigma }⟶{\Re }_{\Sigma }$

   $\widehat{•}:{\Re }_{\Sigma }\times {\Re }_{\Sigma }⟶{\Re }_{\Sigma }$

   $\widehat{*}:{\Re }_{\Sigma }⟶{\Re }_{\Sigma }$

(b)  ∃ an injective (one-to-one) mapping $L:{\Re }_{\Sigma }⟶P\left({\Sigma }^{*}\right)$ s.t.

  (i) $L\left(a\right)=\left\{a\right\}\forall a\in \Sigma$

  (ii) $L\left(\widehat{ϵ}\right)=\left\{ϵ\right\}$

  (iii) $L\left(\widehat{\varnothing }\right)=\varnothing$

  (iv) $L\left(R_1\widehat{\cup }{R}_2\right)=L\left(R_1\right)\cup L\left(R_2\right)\forall R_1,R_2\in {\Re }_{\Sigma }$

  (v) $L\left(R_1\widehat{•}{R}_2\right)=L\left(R_1\right)•L\left(R_2\right)\forall R_1,R_2\in {\Re }_{\Sigma }$

  (vi) $L\left(R_1^{\widehat{*}}\right)={\left(L\left(R_1\right)\right)}^{*}\forall R_1\in {\Re }_{\Sigma }$

${\Re }_{\Sigma }$ is called the set of all regular expressions over the alphabet Σ.

Any member of ${\Re }_{\Sigma }$ is called a regular expression over Σ.

For any regular expression R, $L\left(R\right)$ is called the language described by R.

While $\widehat{\cup },\widehat{•}$ and $\widehat{*}$ are operations in ${\Re }_{\Sigma }$, $\cup ,•$ and * are set operations in $P\left({\Sigma }^{*}\right)$.

When there is no danger of confusion, $\widehat{\cup },\widehat{•}$ and $\widehat{*}$ are usually written same as $\cup ,•$ and *.

While $\widehat{ϵ}$ and $\widehat{\varnothing }$ are regular expressions, ϵ is the empty string and ∅ is the empty language. When there is no danger of confusion, they are all written as ϵ and ∅.

Proposition 10.

Let Σ be a finite alphabet and ${\Re }_{\Sigma }$ be the set of all regular expressions over Σ.

The following statements are true.

(a) $\forall R_1,R_2\in {\Re }_{\Sigma },R_1\cup R_2=R_2\cup R_1$

(b) ∃ regular expressions $\widehat{\Sigma }$ and $\widehat{{\Sigma }^{*}}$ such that $L\left(\widehat{\Sigma }\right)=\Sigma$ and $L\left(\widehat{{\Sigma }^{*}}\right)={\Sigma }^{*}$. When there is no danger of confusion, $\widehat{\Sigma }$ and $\widehat{{\Sigma }^{*}}$ are usually written same as Σ and ${\Sigma }^{*}$.

$<Proof>$

(a) $L(R_1\cup R_2)=L\left(R_1\right)\cup L(R_2$

   $L(R_2\cup R_1)=L\left(R_2\right)\cup L\left(R_1\right)$

  $L\left(R_1\right)\cup L\left(R_2\right)=L\left(R_2\right)\cup L\left(R_1\right)$ from set theory.

  Therefore, $L(R_1\cup R_2)=L(R_2\cup R_1)$

  Therefore, $R_1\cup R_2=R_2\cup R_1$ (L is one-one)

(b) Define $\widehat{\Sigma }={\displaystyle \bigcup _{a\in \Sigma }}a$

  $\widehat{\Sigma }$ is a regular expression by Definition 1.33(a)(i) and 1.33(a)(iv).

  By Definition 1.33(b)(iv)

   $L\left(\widehat{\Sigma }\right)=L\left({\displaystyle \bigcup _{a\in \Sigma }}a\right)=\left({\displaystyle \bigcup _{a\in \Sigma }}L\left(a\right)\right)=\left({\displaystyle \bigcup _{a\in \Sigma }}\left\{a\right\}\right)=\Sigma$

  Define $\widehat{{\Sigma }^{*}}={\left(\widehat{\Sigma }\right)}^{\widehat{*}}$.

  $\widehat{{\Sigma }^{*}}$ is a regular expression by Definition 1.33(a)(vi).

  By Definition 1.33(b)(vi),

   $L\left(\widehat{{\Sigma }^{*}}\right)=L\left({\left(\widehat{\Sigma }\right)}^{\widehat{*}}\right)={\left(L\left(\widehat{\Sigma }\right)\right)}^{*}={\Sigma }^{*}(L\left(\widehat{\Sigma }\right)=\Sigma )$

Example 1.

Find the language described by ${\Sigma }^{*}1{\Sigma }^{*}$ where $\Sigma =\{0,1\}$.

$L\left({\Sigma }^{*}1{\Sigma }^{*}\right)=L\left({\Sigma }^{*}\right)L\left(1\right)L\left({\Sigma }^{*}\right)={\Sigma }^{*}\left\{1\right\}{\Sigma }^{*}=\{w\mid whasatleastone1\}$.

Example 2.

Find the language described by ${(\Sigma \Sigma \Sigma )}^{*}$ where $\Sigma =\{0,1\}$.

$L\left({(\Sigma \Sigma \Sigma )}^{*}\right)={\left(L\right(\Sigma \Sigma \Sigma \left)\right)}^{*}={\left(L\right(\Sigma \left)L\right(\Sigma \left)L\right(\Sigma \left)\right)}^{*}={(\Sigma \Sigma \Sigma )}^{*}$

$={\{xyz\mid x,y,z\in \Sigma \}}^{*}=\{w\mid |w\left|isamultipleofthree\right\}$.

Lemma 2.

If a language is described by a regular expression, then it is regular. That is, if $A=L\left(R\right)$ for some $R\in {\Re }_{\Sigma }$, then $A=L\left(N\right)$ for some finite automaton N.

$<Proof>$

From the formal definition of regular expressions, R is one of the following:

  (i) a for some $a\in \Sigma$

  (ii) $\widehat{ϵ}$

  (iii) $\widehat{\varnothing }$

  (iv) $R_1\widehat{\cup }{R}_2$ for some $R_1,R_2\in {\Re }_{\Sigma }$

  (v) $R_1\widehat{•}{R}_2$ for some $R_1,R_2\in {\Re }_{\Sigma }$

  (vi) $R_1^{\widehat{*}}$ for some $R_1\in {\Re }_{\Sigma }$

In case (i), $L\left(a\right)=\left\{a\right\}$ and $\left\{a\right\}$ can be recognized by the $NFA$ defined as follows:

$N=(\{q_1,q_2\},{\Sigma }_{ϵ},\delta ,q_1,\left\{q_2\right\})$ such that $\delta (q_1,a)=\left\{q_2\right\}$, $\delta (q,b)=\varnothing \forall q\ne q_1,b\ne a$.

In case (ii), $L\left(ϵ\right)=\left\{ϵ\right\}$ and $\left\{ϵ\right\}$ can be recognized by the following $NFA$:

$N=(\left\{q_1\right\},{\Sigma }_{ϵ},\delta ,q_1,\left\{q_1\right\})$, where $\delta (q_1,b)=\varnothing \forall b\ne ϵ$ and $\delta (q_1,ϵ)=\left\{q_1\right\}$.

In case (iii), $L(\varnothing )=\varnothing$, which is recognized by the following $NFA$:

$N=(\left\{q\right\},{\Sigma }_{ϵ},\delta ,q,\varnothing )$ where $\delta (q,b)=\varnothing \forall b\in {\Sigma }_{ϵ}$.

In cases (iv), (v) and (vi), R is repeated operations of $\widehat{\cup },\widehat{•}$ and $\widehat{*}$ on a, ϵ and ∅. Since we have shown above $L\left(a\right)$, $L\left(ϵ\right)$ and $L(\varnothing )$ are regular and we have proved before that regular languages are closed under $\cup ,•$ and * , $L\left(R\right)$ is regular.

Definition 12.

A generalized nondeterministic finite automaton (denoted by $GNFA$) has all the properties as described in Theorem 1.28 and is a 5-tuple, $(Q,\Sigma ,\delta ,q_{start},\left\{q_{accept}\right\})$ where

  (i) Q is a finite set of states;

  (ii) Σ is a finite alphabet;

  (iii) $\delta :(Q\setminus \left\{q_{accept}\right\})\times (Q\setminus \left\{q_{start}\right\})⟶{\Re }_{\Sigma }$ is the transition function;

  (iv) $q_{start}$ is the start state; and

  (v) $q_{accept}$ is the accept state.

A $GNFA$ accepts a string $w\in {\Sigma }^{*}$, if $w=w_1{w}_2\dots w_n$, where each $w_i$ is in ${\Sigma }^{*}$ and a sequence of states $q_0,q_1,q_2,\dots q_n$ exist such that

(1) $q_0=q_{start}$;

(2) $q_n=q_{accept}$; and

(3) For each i, $w_i\in L\left(R_i\right)$ where

  $R_i=\delta (q_{i-1},q_i)$ and $L\left(R_i\right)$ is the language described by expression $R_i$.

If we write $q_i\stackrel{R,\delta }{⟶}{q}_j$ instead of $\delta (q_i,q_j)=R$, the definition of acceptance can be written as

$q_{start}=q_0\stackrel{R_1,\delta }{⟶}{q}_1\stackrel{R_2,\delta }{⟶}{q}_2\dots \stackrel{R_n,\delta }{⟶}{q}_n=q_{accept}$ with $w_i\in L\left(R_i\right)$ for $i=1,2,\dots ,n$.

Lemma 3.

Every $NFA$ can be converted into an equivalent $GNFA$.

$<Proof>$

Because of Theorem 1.28, we can start with an $NFA$ defined as follows.

$N=(Q,\Sigma ,\delta ,q_{start},\left\{q_{accept}\right\})$ where

$q_{start}\ne q_{accept}$; $\delta (q_{accept},a)=\varnothing \forall a\in {\Sigma }_{ϵ}$; and $q_{start}\notin \delta (q,a)\forall a\in {\Sigma }_{ϵ},q\in Q$.

Define $GNFA$, $N_G$ as follows:

$N_G=(Q,\Sigma ,{\delta }_G,q_{start},\left\{q_{accept}\right\})$ where

${\delta }_G:(Q\setminus \left\{q_{accept}\right\})\times (Q\setminus \left\{q_{start}\right\})⟶{\Re }_{\Sigma }$ such that:

$\forall (q_i,q_j)\in (Q\setminus \left\{q_{accept}\right\})\times (Q\setminus \left\{q_{start}\right\})$

${\delta }_G(q_i,q_j)=R_{i,j}$ where

$R_{i,j}={\displaystyle \bigcup _{w\in S_{i,j}}}w$ ; and

$S_{i,j}=\{w\in {\Sigma }^{*}\mid q_i\stackrel{w,\widehat{\delta }}{⟶}{q}_j\}$.

Note that if $i=j$, ${\delta }_G(q_i,q_i)=R_{i,i}$, $S_{i,i}=\{w\in {\Sigma }^{*}\mid q_i\stackrel{w,\widehat{\delta }}{⟶}{q}_i\}$; and

$R_{i,i}={\displaystyle \bigcup _{w\in S_{i,i}}}{w}^{*}$

$\forall (q_i,q_j),S_{i,j}$ is unique and therefore $R_{i,j}$ is unique.

Since w is the concatenation of symbols from Σ, and every symbol in Σ is a regular expression, w is a regular expression.

Therefore, $R_{i,j}={\displaystyle \bigcup _{w\in S_{i,j}}}w$ is a regular expression.

Therefore, ${\delta }_G(q_i,q_j)=R_{i,j}$ is well defined.

Claim 1. For any string w in ${\Sigma }^{*}$, $L\left(w\right)=\left\{w\right\}$.

< Proof of Claim 1 >

$L\left(w\right)=L(a_1{a}_2\dots a_n)$ where $a_i\in \Sigma$

   $=L\left(a_1\right)L\left(a_2\right)\dots L\left(a_n\right)$

   $=\left\{a_1\right\}\left\{a_2\right\}\dots \left\{a_n\right\}$

   $=\{a_1{a}_2\dots a_n\}$

   $=\left\{w\right\}$

Claim 2. $\forall w\in {\Sigma }^{*}$, N accepts $w\iff N_G$ accepts w.

< Proof of Claim 2 >

For forward direction $"\Rightarrow "$

Let N accepts w where $w=w_1{w}_2\dots w_n$, $n\ge 1$, and each $w_i$ is in ${\Sigma }^{*}$ for $1\le i\le n$.

By theorem of acceptance, $\exists q_0,q_1,q_2,\dots q_n\in Q$ such that

$q_{start}=q_0\stackrel{w_1,\widehat{\delta }}{⟶}{q}_1\stackrel{w_2,\widehat{\delta }}{⟶}{q}_2\dots q_{n-1}\stackrel{w_n,\widehat{\delta }}{⟶}{q}_n=q_{accept}$.

Since $q_{i-1}\stackrel{w_i,\widehat{\delta }}{⟶}{q}_i,w_i\in S_{i-1,i}$.

By definition of ${\delta }_G$,

${\delta }_G(q_{i-1},q_i)=R_{i-1,i}={\displaystyle \bigcup _{w\in S_{i-1,i}}}w$

$L\left({\delta }_G(q_{i-1},q_i)\right)$

$=L\left(R_{i-1,i}\right)$

$=L\left({\displaystyle \bigcup _{w\in S_{i-1,i}}}w\right)$

$={\displaystyle \bigcup _{w\in S_{i-1,i}}}L\left(w\right)$

$={\displaystyle \bigcup _{w\in S_{i-1,i}}}\left\{w\right\}$(By Claim 1)

$=S_{i-1,i}$.

Since $w_i\in S_{i-1,i}$, $w_i\in L\left(R_{i-1,i}\right)$.

Since $q_{i-1}\stackrel{R_{i-1,i},{\delta }_G}{⟶}{q}_i\forall i=1,2,\dots n$,

$q_{start}=q_0\stackrel{R_{0,1},{\delta }_G}{⟶}{q}_1\stackrel{R_{1,2},{\delta }_G}{⟶}{q}_2\dots q_{i-1}\stackrel{R_{i-1,i},{\delta }_G}{⟶}{q}_i\dots q_{n-1}\stackrel{R_{n-1,n},{\delta }_G}{⟶}{q}_n=q_{accept}$.

$N_G$ accepts w.

Conversely, if $N_G$ accepts w for $w=w_1{w}_2\dots w_n$, $n\ge 1$, and each $w_i$ is in ${\Sigma }^{*}$,

$\exists q_0,q_1,q_2,\dots q_n\in Q$ such that

$q_{start}=q_0\stackrel{R_{0,1},{\delta }_G}{⟶}{q}_1\stackrel{R_{1,2},{\delta }_G}{⟶}{q}_2\dots q_{i-1}\stackrel{R_{i-1,i},{\delta }_G}{⟶}{q}_i\dots q_{n-1}\stackrel{R_{n-1,n},{\delta }_G}{⟶}{q}_n=q_{accept}$

with $w_i\in L\left(R_{i-1,i}\right)\forall i\in \{1,2,3,\dots n\}$,

$R_{i-1,i}={\displaystyle \bigcup _{w\in S_{i-1,i}}}w$

and

$S_{i-1,i}=\{w\in {\Sigma }^{*}\mid q_{i-1}\stackrel{w,\widehat{\delta }}{⟶}{q}_i\}$

$L\left(R_{i-1,i}\right)$

$=L\left({\displaystyle \bigcup _{w\in S_{i-1,i}}}w\right)$

$={\displaystyle \bigcup _{w\in S_{i-1,i}}}L\left(w\right)$

$={\displaystyle \bigcup _{w\in S_{i-1,i}}}\left\{w\right\}$(By Claim 1)

$=S_{i-1,i}$.

$\forall i\in \{1,2,3,\dots n\}$,

$w_i\in L\left(R_{i-1,i}\right)$

$\Rightarrow w_i\in S_{i-1,i}$

$\Rightarrow q_{i-1}\stackrel{w_i,\widehat{\delta }}{⟶}{q}_i$(Definition of $S_{i,j}$)

Therefore, $q_{start}=q_0\stackrel{w_1,\widehat{\delta }}{⟶}{q}_1\stackrel{w_2,\widehat{\delta }}{⟶}{q}_2\dots q_{n-1}\stackrel{w_n,\widehat{\delta }}{⟶}{q}_n=q_{accept}$.

Therefore, N accepts $w=w_1{w}_2\dots w_n$.

N and $N_G$ are equivalent and the Lemma is proved.

Lemma 4.

Every $GNFA$ of n states ($n\ge 2$) can be reduced to an equivalent $GNFA$ of 2 states.

$<Proof>$

This lemma can be proved by induction on n.

It is trivial that the statement is true for $n=2$.

Assume that the statement is true for $n=k\ge 2$.

Let $G=(Q,\Sigma ,\delta ,q_{start},\left\{q_{accept}\right\})$ be a $GNFA$ with $k+1$ states.

$\exists q_{rip}\in Q\setminus \{q_{start},q_{accept}\}$ because $k+1\ge 3$.

Construct $G^{\prime }=(Q^{\prime },\Sigma ,{\delta }^{\prime },q_{start},\left\{q_{accept}\right\})$ such that

$Q^{\prime }=Q\setminus \left\{q_{rip}\right\}$

$\forall (q_i,q_j)\in (Q\setminus \left\{q_{accept}\right\})\times (Q\setminus \left\{q_{start}\right\})$,

${\delta }^{\prime }(q_i,q_j)=\delta (q_i,q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_j)\cup \delta (q_i,q_j)$.

Therefore, $Q^{\prime }$ is a $GNFA$ with k states.

Let G accept $w=w_1{w}_2\dots w_n$ where each $w_i\in {\Sigma }^{*}$.

$\exists q_0,q_1,q_2,\dots q_n\in Q$ such that

$q_{start}=q_0\stackrel{R_1,\delta }{⟶}{q}_1\stackrel{R_2,\delta }{⟶}{q}_2\dots q_{i-1}\stackrel{R_i,\delta }{⟶}{q}_i\dots q_{n-1}\stackrel{R_n,\delta }{⟶}{q}_n=q_{accept}$; and

$w_i\in L\left(R_i\right)=L\left(\delta (q_{i-1},q_i)\right)$.

If none of $q_0,q_1,q_2,\dots q_n$ is $q_{rip}$, then they are all in $Q^{\prime }$.

Also,

$w_i\in L\left(\delta (q_{i-1},q_i)\right)$

$\Rightarrow w_i\in L\left(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_i)\right)\cup L\left(\delta (q_{i-1},q_i)\right)$

$\Rightarrow w_i\in L\left(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_i)\cup \delta (q_{i-1},q_i)\right)$

$\Rightarrow w_i\in L\left({\delta }^{\prime }(q_{i-1},q_i)\right)$

$\Rightarrow w_i\in L\left(R_i^{\prime }\right)$ where $R_i^{\prime }={\delta }^{\prime }(q_{i-1},q_i)$

$q_{start}=q_0\stackrel{R_1^{\prime },{\delta }^{\prime }}{⟶}{q}_1\stackrel{R_2^{\prime },{\delta }^{\prime }}{⟶}{q}_2\dots q_{i-1}\stackrel{R_i^{\prime },{\delta }^{\prime }}{⟶}{q}_i\dots q_{n-1}\stackrel{R_n\prime ,{\delta }^{\prime }}{⟶}{q}_n=q_{accept}$

with $w_i\in L\left(R_i^{\prime }\right)$.

Therefore, $G^{\prime }$ accepts $w=w_1{w}_2\dots w_n$.

If ∃ some q’s in the sequence $q_0,q_1,q_2,\dots q_n$ which are $q_{rip}$,

let $q_i$ be the first such $q_{rip}$ and $q_j$ be the first state in the sequence after $q_i$ such that $q_j\ne q_{rip}$.

$q_{i-1}\stackrel{R_i}{⟶}{q}_i=q_{rip}\stackrel{R_{i+1}}{⟶}{q}_{rip}\dots q_{rip}\stackrel{R_{j-1}}{⟶}{q}_{rip}\stackrel{R_j}{⟶}{q}_j$.

$R_{i+1}=\delta (q_i,q_{i+1})=\delta (q_{rip},q_{rip})\Rightarrow w_{i+1}\in L\left(\delta (q_{rip},q_{rip})\right)$

        ⋮

$R_{j-1}=\delta (q_{j-2},q_{j-1})=\delta (q_{rip},q_{rip})\Rightarrow w_{j-1}\in L\left(\delta (q_{rip},q_{rip})\right)$

$w_{i+1}\dots w_{j-1}\in L^{j-i-1}\left(\delta (q_{rip},q_{rip})\right)$

$w_{i+1}\dots w_{j-1}\in L^{*}\left(\delta (q_{rip},q_{rip})\right)$

Let $w_j^{\prime }=w_i{w}_{i+1}\dots w_{j-1}{w}_j$

$w_i\in L\left(\delta (q_{i-1},q_i)\right)$ and $q_i=q_{rip}\Rightarrow w_i\in L\left(\delta (q_{i-1},q_{rip})\right)$

$w_j\in L\left(\delta (q_{j-1},q_j)\right)$ and $q_{j-1}=q_{rip}\Rightarrow w_j\in L\left(\delta (q_{rip},q_j)\right)$

${w_j}^{\prime }\in L\left(\delta (q_{i-1},q_{rip})\right)L^{*}\left(\delta (q_{rip},q_{rip})\right)L\left(\delta (q_{rip},q_j)\right)$

$w_j^{\prime }\in L\left(\delta (q_{i-1},q_{rip})\right)L^{*}\left(\delta (q_{rip},q_{rip})\right)L\left(\delta (q_{rip},q_j)\right)\cup L\left(\delta (q_{i-1},q_j)\right)$

Therefore, $w_j^{\prime }\in L\left(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_j)\cup \delta (q_{i-1},q_j)\right)$

$w_j^{\prime }\in L\left({\delta }^{\prime }(q_{i-1},q_j)\right)$

$w_j^{\prime }\in L\left(R_j^{\prime }\right)$ where $R_j^{\prime }={\delta }^{\prime }(q_{i-1},q_j)$

If there are no more $q_{rip}$’s in the sequence,

$q_{start}=q_0\stackrel{R_1^{\prime },{\delta }^{\prime }}{⟶}{q}_1\stackrel{R_2^{\prime },{\delta }^{\prime }}{⟶}{q}_2\dots q_{i-1}\stackrel{R_j^{\prime },{\delta }^{\prime }}{⟶}{q}_j\stackrel{R_{j+1}^{\prime },{\delta }^{\prime }}{⟶}{q}_{j+1}\dots q_{n-1}\stackrel{R_n^{\prime },{\delta }^{\prime }}{⟶}{q}_n=q_{accept}$

is the path of acceptance in $G^{\prime }$ for $(w_1{w}_2\dots w_{i-1})\left(w_j^{\prime }\right)(w_{j+1}\dots w_n)$,

which is the same as $(w_1{w}_2\dots w_{i-1})(w_i{w}_{i+1}\dots w_{j-1}{w}_j)(w_{j+1}\dots w_n)$ because

$w_j^{\prime }=w_i{w}_{i+1}\dots w_{j-1}{w}_j$.

Therefore, $G^{\prime }$ accepts $w=w_1{w}_2\dots w_n$.

If there are some more $q_{rip}$’s in the sequence, repeat the above process until all $q_{rip}$’s are removed and the resulting computation path is the path of acceptance of w in $G^{\prime }$.

Conversely, if $G^{\prime }$ accepts $w=w_1{w}_2\dots w_n$ where $w_i\in {\Sigma }^{*}$,

$\exists q_0,q_1,q_2,\dots q_n\in Q^{\prime }$ such that

$q_{start}=q_0\stackrel{R_1^{\prime },{\delta }^{\prime }}{⟶}{q}_1\stackrel{R_2^{\prime },{\delta }^{\prime }}{⟶}{q}_2\dots q_{i-1}\stackrel{R_i^{\prime },{\delta }^{\prime }}{⟶}{q}_i\dots q_{n-1}\stackrel{R_n^{\prime },{\delta }^{\prime }}{⟶}{q}_n=q_{accept}$

with $w_i\in L\left(R_i^{\prime }\right)$ where $R_i^{\prime }={\delta }^{\prime }(q_{i-1},q_i)$.

Therefore, $w_i\in L(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_i)\cup \delta (q_{i-1},q_i))$

Therefore, $w_i\in L\left(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_i)\right)$ or $w_i\in L\left(\delta (q_{i-1},q_i)\right)$.

If $w_i\in L\left(\delta (q_{i-1},q_i)\right)$,

$q_{start}=q_0\stackrel{R_1,\delta }{⟶}{q}_1\stackrel{R_2,\delta }{⟶}{q}_2\dots q_{i-1}\stackrel{R_i,\delta }{⟶}{q}_i\dots q_{n-1}\stackrel{R_n,\delta }{⟶}{q}_n=q_{accept}$ where $w_i\in L\left(R_i\right)$

is the acceptance path for $w=w_1{w}_2\dots w_n$ in G.

If $w_i\in L\left(\delta (q_{i-1},q_{rip}){\left(\delta (q_{rip},q_{rip})\right)}^{*}\delta (q_{rip},q_i)\right)$,

let $w_i=w_{i1}{w}_{i2}{w}_{i3}$ where

$w_{i1}\in L\left(\delta (q_{i-1},q_{rip})\right)=L\left(R_{i-1,rip}\right)$,

$w_{i2}\in L^{*}\left(\delta (q_{rip},q_{rip})\right)=L^{*}\left(R_{rip,rip}\right)$, and

$w_{i3}\in L\left(\delta (q_{rip},q_i)\right)=L\left(R_{rip,i}\right)$.

$\exists m\ge 0$ such that $w_{i2}\in L^m\left(\delta (q_{rip},q_{rip})\right)$.

$w_{i2}=w_{i2}\left(1\right)w_{i2}\left(2\right)\dots w_{i2}\left(m\right)$ where each $w_{i2}\left(j\right)\in L\left(\delta (q_{rip},q_{rip})\right)=L\left(R_{rip,rip}\right)$.

$q_{i-1}\stackrel{R_{i-1,rip},\delta }{⟶}{q}_{rip}\stackrel{R_{rip,rip},\delta }{⟶}{q}_{rip}\dots q_{rip}\stackrel{R_{rip,i},\delta }{⟶}{q}_i$

is a computation path in G for $w_{i1}{w}_{i2}{w}_{i3}=w_i$.

This is true for all $1\le i\le n$.

Therefore, there is a computation path in G from $q_0$ to $q_n$ for $w_1{w}_2\dots w_n=w$.

Therefore, G accepts $w=w_1{w}_2\dots w_n$.

So G and $G^{\prime }$ are equivalent.

Since $G^{\prime }$ has k states, by induction hypothesis, $G^{\prime }$ can be reduced to an equivalent $GNFA$ of 2 states.

Hence, G can be reduced to an equivalent $GNFA$ of 2 states.

This completes the proof.

Lemma 5.

If an $NFA$, $N=(Q,\Sigma ,\delta ,q_0,F)$ is equivalent to a 2-state $GNFA$, $N_G=(Q,\Sigma ,{\delta }_G,q_{start},\left\{q_{accept}\right\})$, then $L\left(N\right)=L\left(R\right)$ where $R={\delta }_G(q_{start},q_{accept})$.

$<Proof>$

$w\in L\left(N\right)$

$\iff N$ accepts w

$\iff N_G$ accepts $w$ (N and $N_G$ are equivalent.)

$\iff w\in L\left(R\right)$ ($R={\delta }_G(q_{start},q_{accept})$)

By Lemmas 1.39, 1.40, 1.41, we have the following conclusion:

Lemma 6.

If a language is regular, it is described by a regular expression.

By Lemma 1.37 and Lemma 1.42, we have the following theorem.

Theorem 12.

A language is regular iff some regular expression describes it.

7. Pumping Lemma

Theorem 13.

- Pumping Lemma

Let A be a language.

Let $\left(S\right)$ denote the following statement:

∃ a number p (the pumping length) where, if s is any string in A of length at least p, then s may be divided into three pieces, $s=xyz$, satisfying the following conditions:

(1) For each $i\ge 0$, $x{y}^{i}z\in A$,

(2) $\left|y\right|>0$, and

(3) $\left|xy\right|\le p$.

The Pumping Lemma states that A is regular $\Rightarrow \left(S\right)$.

$<Proof>$

Since A is regular, there exists a finite automaton $M=(Q,\Sigma ,\delta ,q_0,F)$ that recognizes A.

That is, $A=L\left(M\right)$.

Let p be the number of states in M.

Let $s=s_1{s}_2\dots s_n$ where each $s_i\in \Sigma$ and $0\le p\le n$.

$\exists r_0,r_1,\dots r_n\in Q$, such that

$q_0=r_0\stackrel{s_1,\delta }{⟶}{r}_1\stackrel{s_2,\delta }{⟶}{r}_2\dots r_{n-1}\stackrel{s_n,\delta }{⟶}{r}_n,r_n\in F$.

Since $p\le n,q_0=r_0\stackrel{s_1,\delta }{⟶}{r}_1\stackrel{s_2,\delta }{⟶}{r}_2\dots r_{p-1}\stackrel{s_p,\delta }{⟶}{r}_p$ is a sub path with $p+1$ states.

Since M has only p states, by the pigeonhole principle, $\exists k,l$ such that $0\le k<l\le p$ and $r_k=r_l$.

Let $x=s_1{s}_2\dots s_k,y=s_{k+1}{s}_{k+2}\dots s_l$ and $z=s_{l+1}{s}_{l+2}\dots s_n$.

Therefore, $r_0\stackrel{x,\widehat{\delta }}{⟶}{r}_k\stackrel{y,\widehat{\delta }}{⟶}{r}_l\stackrel{z,\widehat{\delta }}{⟶}{r}_n$.

Since $r_k=r_l$, $r_k\stackrel{y^i,\widehat{\delta }}{⟶}{r}_l\forall i\ge 0$.

Therefore, $r_0\stackrel{x,\widehat{\delta }}{⟶}{r}_k\stackrel{y^i,\widehat{\delta }}{⟶}{r}_l\stackrel{z,\widehat{\delta }}{⟶}{r}_n$ with $r_n\in F$.

Therefore, M accepts $x{y}^{i}z$.

Therefore, $x{y}^{i}z\in A$.

Since $k<l$, $\left|y\right|>0$.

$\left|xy\right|=\left|x\right|+\left|y\right|=k+l-k=l\le p$.

This completes the proof of the Pumping Lemma.

Theorem 14.

- Pumping Lemma (contra positive form)

$\neg \left(S\right)\Rightarrow A$ is not regular where

$\neg \left(S\right)$ is equivalent to:

$\forall p\ge 1,\exists s\in A$ with $\left|s\right|\ge p$ such that whenever $s=xyz$, at least one of the conditions $\left(1\right),\left(2\right),$ or $\left(3\right)$ cannot be satisfied.

The contra positive form of the Pumping Lemma is used to prove a language is not regular. The general strategy is to find an $s\in A$ with $\left|s\right|\ge p$ for any given $p\ge 1$ so that whenever s is broken into $s=xyz$, at least one of the conditions of $\left(1\right),\left(2\right),$ or $\left(3\right)$ must be false. This can be usually accomplished by showing one of the following:

(i) Condition 1 alone is false.

(ii) Condition 3 $\Rightarrow \neg$(Condition 1)

(iii) (Condition 2 and Condition 3) $\Rightarrow \neg$(Condition 1).

Example 3.

Show that $A=\{0^n{1}^n\mid n\ge 0\}$ is not regular.

The strategy is to create an s that will force y to contain all 0’s or all 1’s so that when y is pumped indefinitely, $x{y}^{i}z$ will contain too many 0’s or 1’s to make it impossible for $x{y}^{i}z$ to remain in A.

Since Condition 3 requires $\left|xy\right|\le p$, a prefix of $0^p$ in s will achieve that purpose.

Formally, we make the argument as follows.

$\forall p\ge 1$, let $s=0^p{1}^p$.

$s\in A$ and $\left|s\right|\ge p$.

If $s=xyz$, then $xyz=0^p{1}^p$.

Condition 3

$\Rightarrow \left|xy\right|\le p$

$\Rightarrow xy$ consists of only 0’s

$\Rightarrow y$ consists of only 0’s.

$\left|xyyz\right|=\left|xyz\right|+\left|y\right|$.

Since Condition 2 requires $\left|y\right|>0$, $xyyz$ adds a positive number of 0’s to $xyz$.

Since $xyz$ has equal numbers of 0’s and 1’s, $xyyz$ must have more 0’s than 1’s and hence is not in A.

Therefore, (Condition 2 + Condition 3) $\Rightarrow \neg$(Condition 1) and hence A is not regular.

Example 4.

Show that $A=\{ww\mid w\in {\{0,1\}}^{*}\}$ is not regular.

The strategy is to create an s with some leading 0’s on the left, say $0^m$ but we also want to make sure that $0^m$ is long enough to force $xy$ to contain all 0’s in it so that when y is pumped up indefinitely, it will create too many 0’s to make it impossible for $s=ww$.

Since Condition 3 requires $\left|xy\right|\le p$, we want to make $m\ge p$.

A natural candidate for s is therefore $0^p{10}^{p}1$.

To prove that this construction works, however, requires some algebraic manipulation.

Formally, we make the argument as follows.

$\forall p\ge 1$, take $s=0^p{10}^{p}1$.

If $s=xyz$, then $xyz=0^p{10}^{p}1$.

Condition 3

$\Rightarrow \left|xy\right|\le p$

$\Rightarrow xy$ consists of only 0’s

$\Rightarrow y$ consists of only 0’s.

Let $x{y}^{i}z=0^{p^{\prime }}{10}^{p}1$ where $p^{\prime }-p=(i-1)\left|y\right|$ or $p^{\prime }=p+(i-1)\left|y\right|$.

For $i>3$, $p^{\prime }>p+(3-1)$ ($\left|y\right|\ge 1$ by Condition 2)

Therefore, $p^{\prime }>p+2$ for $i>3$.

Assume for contradiction that $\forall i\ge 0$, $x{y}^{i}z\in A$.

That is, $x{y}^{i}z=0^{p^{\prime }}{10}^{p}1=ww$.

For all $i>3$,

$\left|w\right|$

$={\displaystyle \frac{|0^{p^{\prime }}{10}^{p}1|}{2}}$

$={\displaystyle \frac{p^{\prime }+p+2}{2}}$

$>{\displaystyle \frac{p+2+p+2}{2}}$ ($p^{\prime }>p+2$ for $i>3$)

$=p+2$

Therefore, $\left|w\right|>|{10}^{p}1|$.

This implies w consists of at least two 1’s.

On the other hand, $p^{\prime }+p+2=2\left|w\right|$.

$p^{\prime }-\left|w\right|=\left|w\right|-(p+2)>0$

$p^{\prime }>\left|w\right|$

This implies w must consist of all 0’s.

This leads to a contradiction.

Therefore, (Condition 2 + Condition 3) $\Rightarrow \neg$(Condition 1) and hence A is not regular.

Example 5.

Show that $A=\{1^{n^2}\mid n\ge 0\}$ is not regular.

The idea behind this problem is every time we pump up y, we increase the length of s by an amount of $\left|y\right|$ which is bounded by p and p is fixed. On the other hand, s has to be the square of a natural number and the difference between two consecutive squares, say $n^2$ and ${(n+1)}^2$ will grow to infinity as n goes to infinity. In this case, we don’t have to worry about how to create more 0’s in s so as to outnumber the 1’s or vice versa. This particular nature of s will automatically lead to a contradiction to Condition 1 as $\left|s\right|$ grows to infinity.

Proving this to work requires some algebraic manipulation.

The formal argument is made as follows.

$\forall p\ge 1$, take $s=1^{p^2}$

$p\ge 1$

$\Rightarrow p(p-1)\ge 0$

$\Rightarrow p^2\ge p$

$\Rightarrow |1^{p^2}|\ge |1^p|=p$

Therefore, $\left|s\right|\ge p$.

Assume for contradiction that Condition 1 is true.

That is, $\forall i\ge 0$, $x{y}^{i}z\in A$.

Both $x{y}^{i}z$ and $x{y}^{i+1}z$ are in A.

Let $x{y}^{i}z=1^{n^2}$ and $x{y}^{i+1}z=1^{m^2}$ where m and n are positive integers.

$|x{y}^{i}z|=n^2$ and $|x{y}^{i+1}z|=m^2$.

By Condition 2,

$\left|y\right|\ge 1$

$\Rightarrow |y^{i+1}|>|y^i|$

$\Rightarrow |x{y}^{i+1}z|>|x{y}^{i}z|$

$\Rightarrow m^2>n^2$

$\Rightarrow m>n$

$\Rightarrow m\ge n+1$

By Condition 3, $\left|xy\right|\le p\Rightarrow \left|y\right|\le p$.

Therefore, $|x{y}^{i+1}z|-|x{y}^{i}z|=|y|\le p$.

Therefore, $m^2-n^2\le p$.

${(n+1)}^2-n^2\le m^2-n^2\le p$.

$2n+1\le p$.

$n\le {\displaystyle \frac{p-1}{2}}\dots \dots \left(1\right)$ where $\left(1\right)$ is true for all i.

On the other hand,

Condition 2 $\Rightarrow \left|y\right|\ge 1\Rightarrow |y^i|\ge i$.

$n^2=\left|x\right|+|y^i|+|z|\ge |y^i|\ge i$.

$n\ge \sqrt{i}$ for all i.

For $i>{\displaystyle \frac{{(p-1)}^2}{4}}$, $\sqrt{i}>{\displaystyle \frac{p-1}{2}}$ and $n>{\displaystyle \frac{p-1}{2}}$

This contradicts $\left(1\right)$ which is true for all i.

Therefore, (Condition 2 + Condition 3) $\Rightarrow \neg$(Condition 1) and hence A is not regular.

8. Myhill-Nerode Theorem

Definition 13.

$\forall x,y\in {\Sigma }^{*},L\subset {\Sigma }^{*}$,

we say that x and y are indistinguishable by L iff $\forall z\in {\Sigma }^{*},xz\in L\iff yz\in L$.

We say that x and y are distinguishable by L iff there exists $z\in {\Sigma }^{*}$ such that exactly one of $xz$ and $yz$ is in L.

If x and y are indistinguishable by L, we write $x{\equiv }_{L}y$.

Proposition 11.

${\equiv }_L$ is an equivalence relation.

$<Proof>$

$\forall x\in L,xz\in L\iff xz\in L\forall z\in {\Sigma }^{*}$

$x{\equiv }_{L}x$

${\equiv }_L$ is reflexive.

$\forall x,y\in L$,

$x{\equiv }_{L}y$

$\Rightarrow (\forall z\in {\Sigma }^{*},xz\in L\iff yz\in L)$

$\Rightarrow (\forall z\in {\Sigma }^{*},yz\in L\iff xz\in L)$

$\Rightarrow y{\equiv }_{L}x$

${\equiv }_L$ is symmetric.

$\forall x,y,w\in {\Sigma }^{*}$,

$\left(x{\equiv }_{L}y\right)\wedge \left(y{\equiv }_{L}w\right)$

$\Rightarrow (\forall z\in {\Sigma }^{*},xz\in L\iff yz\in L)\wedge (\forall z\in {\Sigma }^{*},yz\in L\iff wz\in L)$

$\Rightarrow (\forall z\in {\Sigma }^{*},xz\in L\iff wz\in L)$

$\Rightarrow x{\equiv }_{L}w$

${\equiv }_L$ is transitive.

Proposition 12.

${\equiv }_L$ is right congruence. That is $x{\equiv }_{L}y\Rightarrow xa{\equiv }_{L}ya\forall a\in \Sigma$.

$<Proof>$

$\forall z\in {\Sigma }^{*},a\in \Sigma$,

$xaz\in L\iff yaz\in L$ ($x{\equiv }_{L}y$)

$xa{\equiv }_{L}ya$ (Definition of ${\equiv }_L$)

Proposition 13.

$\forall x,y\in {\Sigma }^{*},\left(x{\equiv }_{L}y\right)\Rightarrow (x\in L\iff y\in L)$

$<Proof>$

Take $z=ϵ$.

$xϵ\in L\iff yϵ\in L$

Therefore, $x\in L\iff y\in L$.

Theorem 15.

- Myhill-Nerode Theorem

Let $L\subset {\Sigma }^{*},X\subset {\Sigma }^{*}$.

X is said to be pairwise distinguishable by L iff every two distinct strings in X are

distinguishable by L.

The index of L is defined as

$IndexL=max\left\{\right|X|\mid XispairwisedistinguishablebyL\}$.

The following statements are true:

$\left(a\right)$ If L is recognized by a $DFA$ with k states, L has an index at most k.

$\left(b\right)$ If the index of L is a finite number k, it is recognized by a $DFA$ with k states.

$\left(c\right)$L is regular iff it has finite index. Moreover, its index is the size of the smallest $DFA$

recognizing it.

$<Proof>$

$\left(a\right)$ Let $M=(Q,\Sigma ,\delta ,q_0,F)$ be a $DFA$ with k states that recognizes L.

  Assume for contradiction that L has an index greater than k.

  $\exists X$ (pairwise distinguishable by L) that has more than k members.

  Let $s_1,s_2,s_3\dots s_{k+1}$ be $k+1$ distinct and pairwise distinguishable members in X.

  $\widehat{\delta }(q_0,s_1),\widehat{\delta }(q_0,s_2),\widehat{\delta }(q_0,s_3),\dots \widehat{\delta }(q_0,s_{k+1})$ are $k+1$ states in Q.

  Since $\left|Q\right|=k$, by the pigeonhole principle, there are $i,j$ where $1\le i<j\le k+1$ s.t.

  $\widehat{\delta }(q_0,s_i)=\widehat{\delta }(q_0,s_j)$.

  $\forall z\in {\Sigma }^{*}$,

   $s_{i}z\in L$

  $\iff \widehat{\delta }(q_0,s_{i}z)\in F$ (M recognizes L)

  $\iff \widehat{\delta }(\widehat{\delta }(q_0,s_i),z)\in F$ (Proposition 1.14)

   $\iff \widehat{\delta }(\widehat{\delta }(q_0,s_j),z)\in F$

  $\iff \widehat{\delta }(q_0,s_{j}z)\in F$ (Proposition 1.14)

  $\iff s_{j}z\in L$ (M recognizes L)

  Therefore, $s_i{\equiv }_L{s}_j$ (Definition of ${\equiv }_L$)

  This contradicts the assumption that X is pairwise distinguishable by L.

$\left(b\right)$ Let $X=\{s_1,s_2\dots ,s_k\}$ be pairwise distinguishable by L.

  Claim 1. $Index$$L\ge 2\Rightarrow L\ne \varnothing$ and hence $L=\varnothing \Rightarrow Index$$L=1$.

  <Proof of Claim 1>

  $Index$$L\ge 2$

  $\Rightarrow \exists X$ (pairwise distinguishable by L) that has at least 2 members.

  $\Rightarrow \exists s_i,s_j\in X$ where $s_i\ne s_j$ and $s_i,s_j$ are distinguishable by L.

  $\Rightarrow \exists z\in {\Sigma }^{*}$ s.t. $s_{i}z\in L$ and $s_{j}z\notin L$ or vice versa.

  $\Rightarrow L\ne \varnothing$.

  Since $L=\varnothing \Rightarrow Index$$L<2$ or $Index$$L=1$, $Index$L is defined to be 1 whenever $L=\varnothing$.

  Claim 2. $\forall w\in {\Sigma }^{*}$, there is one and only one $s_w\in X$ s.t. $w{\equiv }_L{s}_w$. Hence by taking $w=ϵ$,

  there is one and only one $s_{ϵ}\in X$ s.t. $ϵ{\equiv }_L{s}_{ϵ}$.

  <Proof of Claim 2>

  Either $w\in X$ or $w\notin X$.

  If $w\in X,\exists s_i\in X$ s.t. $w=s_i$.

  Call this $s_w$ so that $w=s_i=s_w$.

  Since ${\equiv }_L$ is reflexive, it follows that $w{\equiv }_L{s}_w$.

  If $w\notin X$, w must be indistinguishable with a member of X otherwise it will contradict

  the assumption that $Index$$L=k$.

  Therefore, $w{\equiv }_L{s}_w$ for some $s_w\in X$.

  Either case, $w{\equiv }_L{s}_w$ for some $s_w\in X$.

  If there is another $s_w^{\prime }\in X$ s.t. $w{\equiv }_L{s}_w^{\prime }$, then $s_w{\equiv }_L{s}_w^{\prime }$ because ${\equiv }_L$ is transitive.

  This contradicts the assumption that X is pairwise distinguishable by L.

  Therefore, $s_w$ is unique.

  Claim 3. If $L\ne \varnothing$ then $L\cap X\ne \varnothing$

  <Proof of Claim 3>

   $L\ne \varnothing \Rightarrow \exists w\in L$

  By Claim 2, there is one and only one $s_w\in X$ s.t. $w{\equiv }_L{s}_w$.

  By Proposition 1.52, $w\in L\iff s_w\in L$.

  Therefore, $s_w\in L\cap X$.

  Therefore, $L\cap X\ne \varnothing$.

  This completes proof of Claim 3.

  If Index $L=k=1$, $L=\varnothing$ which is recognized by the one-state $DFA$, $M=\left(\right\{q\},\Sigma ,\delta ,q,\varnothing )$

  where $\delta (q,b)=\varnothing \forall b\in \Sigma$.

  If $Index$$L=k\ge 2$,

  $\exists X=\{s_1,s_2,s_3\dots s_k\}$ where X is pairwise distinguishable by L.

  Let $Q=\{q_1,q_2,q_3\dots q_k\}$

  Let $f:X⟶Q$ such that $f\left(s_i\right)=q_i\forall i$ with $1\le i\le k$

  f is bijective (one-one and onto).

  $\forall q_i\in Q,\exists$ a unique $s_i\in X$ s.t. $f\left(s_i\right)=q_i$ since f is bijective.

  $\forall a\in \Sigma ,\exists$ a unique $s_j\in X$ s.t. $s_{i}a{\equiv }_L{s}_j$ by Claim 2.

  Since f is a bijective mapping, there is a unique $q_j$ such that $f\left(s_j\right)=q_j$.

  Let $M=(Q,\Sigma ,\delta ,q_0,F)$ where

  $\delta :Q\times \Sigma ⟶Q$ s.t. $\delta (q_i,a)=q_j$ where

  $a\in \Sigma ,q_i,q_j\in Q$ s.t. $f\left(s_i\right)=q_i,f\left(s_j\right)=q_j$ where $s_i\in X,s_j\in X$ and $s_{i}a{\equiv }_L{s}_j$.

  If there is another $q_k\in Q$ such that $\delta (q_i,a)=q_k,\exists s_k\in X$ such that $f\left(s_k\right)=q_k$ and

  by definition of $\delta ,s_{i}a{\equiv }_L{s}_k$.

  Since ${\equiv }_L$ is transitive, $s_j{\equiv }_L{s}_k$.

  This contradicts that both $s_j$ and $s_k$ are in X and hence must be distinguishable by L.

  Therefore, $\delta (q_i,a)=q_j$ is uniquely defined.

  $q_0=q_{ϵ}$ where $q_{ϵ}=f\left(s_{ϵ}\right)$ and $s_{ϵ}$ is defined in Claim 2.

   $F=\left\{f\right(s)\mid s\in L\cap X\}$

  $F\ne \varnothing$ because of Claim 1 and Claim 3.

  Claim 4. $\forall w\in {\Sigma }^{*}$, $\widehat{\delta }(q_{ϵ},w)=q_i\iff w{\equiv }_L{s}_i$ where $f\left(s_i\right)=q_i$.

  <Proof of Claim 4>

  Claim 4 can be proved by induction on $\left|w\right|$.

  For $w=ϵ$, there exists one and only one $s_{ϵ}\in X$ s.t. $ϵ{\equiv }_L{s}_{ϵ}$ by Claim 2.

   $\widehat{\delta }(q_{ϵ},w)=q_i$

  $\iff \widehat{\delta }(q_{ϵ},ϵ)=q_i$ ($w=ϵ$)

  $\iff q_{ϵ}=q_i$ (Definition of 1.4(i))

  $\iff f\left(s_{ϵ}\right)=q_i$ (Definition of $q_{ϵ}$)

  $\iff f\left(s_{ϵ}\right)=f\left(s_i\right)$ (Definition of $q_i$)

  $\iff s_{ϵ}=s_i$ (f is bijective)

  $\iff ϵ{\equiv }_L{s}_i$ ($ϵ{\equiv }_L{s}_{ϵ}$ by Claim 2)

  $\iff w{\equiv }_L{s}_i$ ($w=ϵ$)

  The statement is true for $w=ϵ$.

  Let $\widehat{\delta }(q_{ϵ},wa)=f\left(s_i\right)=q_i$.

  $\delta (\widehat{\delta }(q_{ϵ},w),a)=f\left(s_i\right)=q_i$.

  $\exists q_j$ s.t. $\widehat{\delta }(q_{ϵ},w)=q_j$

  $\exists s_j$ s.t. $f\left(s_j\right)=q_j$ and

  $w{\equiv }_L{s}_j$ (By induction hypothesis)

  $wa{\equiv }_L{s}_{j}a$ (${\equiv }_L$ is right congruence by Proposition 1.51)

   $\delta (q_j,a)=q_i$

  $\Rightarrow s_{j}a{\Rightarrow }_L{s}_i$ (Definition of δ)

  $\Rightarrow wa{\equiv }_L{s}_i$ (${\equiv }_L$ is transitive)

  Conversely, if $wa{\equiv }_L{s}_i$ for some $s_i\in X$,

   $\widehat{\delta }(q_{ϵ},wa)$

   $=\delta (\widehat{\delta }(q_{ϵ},w),a)$

  $=\delta (q_j,a)$ where $q_j=\widehat{\delta }(q_{ϵ},w)$

  By induction hypothesis, $w{\equiv }_L{s}_j$ because $\widehat{\delta }(q_{ϵ},w)=q_j$.

  $wa{\equiv }_L{s}_{j}a$ (Right congruence by Proposition 1.51)

  Let $\delta (q_j,a)=q_k$

  $s_{j}a{\equiv }_L{s}_k$ (By definition of δ)

  $wa{\equiv }_L{s}_k$ (${\equiv }_L$ is transitive)

  $wa{\equiv }_L{s}_i$ (Assumption)

  $s_k=s_i$ (Claim 2)

   $f\left(s_k\right)=f\left(s_i\right)$

   $q_k=q_i$

   $\widehat{\delta }(q_{ϵ},wa)$

   $=\delta (q_j,a)$

   $=q_k$

   $=q_i$

  This completes the proof of Claim 4.

  It remains to prove $L=L\left(M\right)$.

  $\forall w\in L,\exists$ one and only one $s_i\in X$ s.t. $w{\equiv }_L{s}_i$ (By Claim 2)

  $w\in L\iff s_i\in L$ (Proposition 1.52)

  Therefore, $s_i\in L$ ($w\in L$)

  Since $s_i\in L\cap X$ and $q_i=f\left(s_i\right)$, $q_i\in F$ (Definition of F)

  $\widehat{\delta }(q_{ϵ},w)=q_i$ (Claim 4)

  $\widehat{\delta }(q_0,w)=q_i$ ($q_0=q_{ϵ}$)

  M accepts $w$ ($q_i\in F$)

  Conversely, if M accepts w,

  $\widehat{\delta }(q_{ϵ},w)=q_i$ and $q_i\in F$ ($q_0=q_{ϵ}$)

  $w{\equiv }_L{s}_i$ where $q_i=f\left(s_i\right)$ (Claim 4)

  $w\in L\iff s_i\in L$ (Proposition 1.52)

  Since $q_i\in F$ and $q_i=f\left(s_i\right)$,

  $s_i\in L\cap X$ by definition of F.

  Therefore, $s_i\in L$.

  Therefore, $w\in L$.

  $L=L\left(M\right)$ and M has k states.

$\left(c\right)$L is regular

  $\Rightarrow \exists M$ s.t. $L=L\left(M\right)$

  $\Rightarrow Index$$L\le k$ where $k=$ the number of states in $M$ (by (a))

  $\Rightarrow L$ has a finite index

  L has a finite index

  $\Rightarrow Index$$L=k$

  $\Rightarrow L=L\left(M\right)$ for some k-state $DFA$$M$ (by (b))

  $\Rightarrow L$ is regular

  Assume for contradiction that there is a $k^{\prime }$-state $DFA$ accepting L where $k^{\prime }<k$.

  By (a), $Index$$L\le k^{\prime }$.

  This would contradict $k^{\prime }<k=Index$L.

9. An Application of the Myhill-Nerode Theorem

The Myhill-Nerode Theorem can be used to determine whether a language L is regular

or non-regular by determining the number of members in X, the set that is pairwise

distinguishable by L.

Example 6.

Determine if $L=\{a^n{b}^n\mid n\ge 0\}$ is regular.

Consider $X=\{a,a^2,a^3\dots \}$

∀ distinct $x,y\in X,x=a^i,y=a^j$ where $1\le i<j<\infty$

$\exists z=b^i$ such that

$xz=a^i{b}^i\in L$ and $yz=a^j{b}^i\notin L$.

x and y are distinguishable by L. ($x{\neg \equiv }_{L}y$)

X is pairwise distinguishable by L.

$Index$$L\ge \left|X\right|$

$Index$L is infinite.

L is not regular.