Find all functions \(f:\mathbb{R}\rightarrow\mathbb{R}\) such that \(f(x)f(y)=f(xy-1)+xf(y)+yf(x)\), \(\forall x,y\in\mathbb{R}\).

Solution

Replace \(y=0\), we get \(f(x)f(0)=f(-1)+xf(0)\).

If \(f(0)\neq0\), then \(f(x)=x+c\), where \(c\) is a constant. Substituting into the original equation gives a contradiction. Therefore, \(f(0)=0\), and hence \(f(-1)=0\).

Putting \(x=y=1\), we have \(f(1)^2=2f(1)\), so \(f(1)=0\) or \(f(1)=2\).

Putting \(y=-1\), we get \(f(-x-1)=f(x)\).

Replacing \(y\) by \(-y-1\) in the original equation, using \(f(-y-1)=f(y)\) and \(f(xy+x)=f(x(-y-1)-1)\), we obtain

\[ f(xy-1)+yf(x)=f(xy+x)+xf(y)-(y+1)f(x). \]

For \(x\neq0\), let \(y=\frac1x\), then

\[ f(x+1)=\frac{x+2}{x}f(x), \]

or equivalently,

\[ f(x-1)=\frac{x-1}{x+1}f(x). \]

Now replace \(y=1\) in the original equation:

\[ f(x)f(1)=f(x-1)+xf(1)+f(x) \]

Using \(f(x-1)=\frac{x-1}{x+1}f(x)\), we have:

Case 1: \(f(1)=0\), then \(f(x)=0\) for all \(x\in\mathbb{R}\).

Case 2: \(f(1)=2\), then \(f(x)=x(x+1)\).

Both functions satisfy the original equation.

Therefore, the solutions are:

\[ \boxed{f(x)\equiv0\quad\text{or}\quad f(x)=x(x+1),\ \forall x\in\mathbb{R}.} \]

Solution by: Thin Sokkean