I would focus on Acts 20:30 because his audience is identified in Acts 20:17 as the older men. Most likely this was a gathering of overseers from all over Asia Minor. Paul refers to his ministry in Asia Minor (vs. 18). So when he says "from among you yourselves men will rise and speak twisted things" he is referring to those who have the responsibility of teaching. It is wrong to focus on the "ordinary" members of the congregation who may be unsteady or weak or new or have doubts or questions. Apostates are usually found among the leaders. Their motive is the competition for loyalties among members. Apostacy is clearly a power struggle.
Apostates are not people who miss meetings, have low hours etc. They are influential members who are already looked to as leaders.