The alternating direction method of multiplies (ADMM) is an effective method for solving large-scale optimization and nonconvex non-smooth objective. However, the convergence cannot be guaranteed when the objective function is nonconvex and non-smooth. Moreover, the proof of the convergence is more complex when the objective contains coupled function. Many problems in real life are nonconvex in nature, so the research on nonconvex optimization problems is particularly important. Therefore, an improved ADMM algorithm is proposed in this paper. Compared with the original ADMM algorithm, a relaxation factor $\alpha $ is introduced and a generalized alternating direction multiplier method (GADMM) is constructed to solve the non-convex nonseparable optimization problem with linear constraints. Under certain assumptions, by assuming that the augmented Lagrangian function satisfies the K-L inequality, it is proved that the sequence generated by the algorithm converges to the critical point of the augmented Lagrangian function when the penalty parameter is sufficiently large.