# optimization to find the maximum of sum of sigmoids with some constraints

I am having a problem of maximizing a sum of sigmoid functions over different time instants with some constraints.

Considering the standard sigmoid function $$f(x)=\frac{1}{1+e^{-\alpha x}}$$ and it's derivative $$f'(x)=f(x)[1-f(x)]$$

In my case, it is slightly different, the sigmoid function at time instant $$n$$ is defined as $$f_n(x)=\frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}$$, where z>0 and $$\alpha>0$$ are constants and $$y_n>0$$ is defined for every time instant $$n$$.

I need to find $$x$$ that maximize the following:

$$\sum_{n=1}^N \frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}\:\: -c x \:\:$$ such that $$\:\:x \leq X$$, $$\:\:\:(1)$$

where $$c>0$$ is some constant represents a cost value, $$X$$ is another constant represents the maximum value of $$x$$ and $$N$$ is total number of time instants.

One of the solution I was thinking about is to use Lagrangian:

$$L(x,\lambda)=\sum_{n=1}^N \frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}\:\: -c x \:\:- \lambda (x-X)$$

where $$\lambda$$ is the Lagrange multiplier, since we can find the derivative of equation (1) over $$x$$.

I tried this method but after getting $$\frac{\partial L(x,\lambda)}{\partial x}=0 \:\:$$ I could't solve it.

I am not sure this type of optimization problem (1) is solvable or not. And if not is there some type of approximation/relaxation can be used to solve it.

I don't have much experience in optimization problem. So, I was hopping to get some help.

-Thanks

You might get a cue from the following intuition. Suppose the parameters $$\alpha$$, $$y_n$$'s and $$z$$ are such that each of the sigmoids have a very sharp rising edge. In that case, the sum of sigmoids will essentially look like a step function, rising at $$zy_n$$. Needless to say, $$-cx$$ is a line with a negative slope.
Now geometrically, a sum of a step function (with $$zy_n$$'s as transition points) and a negative slope line must have local maximas at $$yz_n$$'s only. Moreover, the local maximas grow in value from the left to the right on the real line. Given an upper bound $$X$$, one chooses as solution the highest transition point lesser than $$X$$.
For the case at hand, you might want to perform a golden section search between $$\frac{zy_{n-1}+zy_{n}}{2}$$ and $$\frac{zy_{n}+zy_{n+1}}{2}$$ $$\forall n$$, and store these values. Also find the value of the function at $$X$$ and finally, choose the maximum among all these. This is only intuitive and the correctness (or otherwise) needs to be established.
{"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 32, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "url": "https://mathoverflow.net/questions/343740/optimization-to-find-the-maximum-of-sum-of-sigmoids-with-some-constraints", "fetch_time": 1611095165000000000, "content_mime_type": "text/html", "warc_filename": "crawl-data/CC-MAIN-2021-04/segments/1610703519784.35/warc/CC-MAIN-20210119201033-20210119231033-00351.warc.gz", "warc_record_offset": 462310887, "warc_record_length": 27695, "token_count": 729, "char_count": 2556, "score": 3.796875, "int_score": 4, "crawl": "CC-MAIN-2021-04", "snapshot_type": "longest", "language": "en", "language_score": 0.896147608757019, "file_path": "/sailhome/yjruan/project/latent-lm/data/processed/finemath-4plus/finemath-4plus/train-00019-of-00064.parquet"}